-- Logs begin at Tue 2022-07-26 13:52:05 UTC, end at Tue 2022-07-26 15:19:15 UTC. -- Jul 26 14:28:42.346251 dv-cinder-851030-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 26 14:28:42.355023 dv-cinder-851030-1-iscsi devstack@keystone.service[80510]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 26 14:28:42.359315 dv-cinder-851030-1-iscsi devstack@keystone.service[80510]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jul 26 14:28:42 2022] *** Jul 26 14:28:42.359315 dv-cinder-851030-1-iscsi devstack@keystone.service[80510]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 26 14:28:42.359315 dv-cinder-851030-1-iscsi devstack@keystone.service[80510]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 26 14:28:42.359315 dv-cinder-851030-1-iscsi devstack@keystone.service[80510]: nodename: dv-cinder-851030-1-iscsi Jul 26 14:28:42.359315 dv-cinder-851030-1-iscsi devstack@keystone.service[80510]: machine: x86_64 Jul 26 14:28:42.359315 dv-cinder-851030-1-iscsi devstack@keystone.service[80510]: clock source: unix Jul 26 14:28:42.359315 dv-cinder-851030-1-iscsi devstack@keystone.service[80510]: pcre jit disabled Jul 26 14:28:42.359315 dv-cinder-851030-1-iscsi devstack@keystone.service[80510]: detected number of CPU cores: 4 Jul 26 14:28:42.359315 dv-cinder-851030-1-iscsi devstack@keystone.service[80510]: current working directory: / Jul 26 14:28:42.359315 dv-cinder-851030-1-iscsi devstack@keystone.service[80510]: detected binary path: /usr/bin/uwsgi-core Jul 26 14:28:42.359315 dv-cinder-851030-1-iscsi devstack@keystone.service[80510]: your processes number limit is 47680 Jul 26 14:28:42.359315 dv-cinder-851030-1-iscsi devstack@keystone.service[80510]: your memory page size is 4096 bytes Jul 26 14:28:42.359315 dv-cinder-851030-1-iscsi devstack@keystone.service[80510]: detected max file descriptor number: 2048 Jul 26 14:28:42.360329 dv-cinder-851030-1-iscsi devstack@keystone.service[80510]: lock engine: pthread robust mutexes Jul 26 14:28:42.360329 dv-cinder-851030-1-iscsi devstack@keystone.service[80510]: thunder lock: enabled Jul 26 14:28:42.360329 dv-cinder-851030-1-iscsi devstack@keystone.service[80510]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 26 14:28:42.360329 dv-cinder-851030-1-iscsi devstack@keystone.service[80510]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 26 14:28:42.399742 dv-cinder-851030-1-iscsi devstack@keystone.service[80510]: Python main interpreter initialized at 0x55a77c2b49c0 Jul 26 14:28:42.399742 dv-cinder-851030-1-iscsi devstack@keystone.service[80510]: python threads support enabled Jul 26 14:28:42.399742 dv-cinder-851030-1-iscsi devstack@keystone.service[80510]: your server socket listen backlog is limited to 100 connections Jul 26 14:28:42.399742 dv-cinder-851030-1-iscsi devstack@keystone.service[80510]: your mercy for graceful operations on workers is 90 seconds Jul 26 14:28:42.400066 dv-cinder-851030-1-iscsi devstack@keystone.service[80510]: mapped 403077 bytes (393 KB) for 2 cores Jul 26 14:28:42.400165 dv-cinder-851030-1-iscsi devstack@keystone.service[80510]: *** Operational MODE: preforking *** Jul 26 14:28:42.400434 dv-cinder-851030-1-iscsi devstack@keystone.service[80510]: *** uWSGI is running in multiple interpreter mode *** Jul 26 14:28:42.400434 dv-cinder-851030-1-iscsi devstack@keystone.service[80510]: spawned uWSGI master process (pid: 80510) Jul 26 14:28:42.400634 dv-cinder-851030-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 26 14:28:42.401308 dv-cinder-851030-1-iscsi devstack@keystone.service[80510]: spawned uWSGI worker 1 (pid: 80522, cores: 1) Jul 26 14:28:42.407425 dv-cinder-851030-1-iscsi devstack@keystone.service[80510]: spawned uWSGI worker 2 (pid: 80523, cores: 1) Jul 26 14:28:42.407425 dv-cinder-851030-1-iscsi devstack@keystone.service[80510]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 26 14:28:47.976906 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 14:28:47.976906 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: from cryptography.utils import int_from_bytes Jul 26 14:28:47.977715 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 14:28:47.977715 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: from cryptography.utils import int_from_bytes Jul 26 14:28:47.978164 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 14:28:47.978164 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: from cryptography.utils import int_from_bytes Jul 26 14:28:47.978272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 14:28:47.978272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: from cryptography.utils import int_from_bytes Jul 26 14:28:48.032336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 26 14:28:48.032522 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 26 14:28:48.032711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] command line args: [] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 26 14:28:48.032806 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 26 14:28:48.033009 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 26 14:28:48.033137 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 26 14:28:48.033233 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ================================================================================ {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 26 14:28:48.033314 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] command line args: [] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 26 14:28:48.033448 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] admin_token = **** {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.033545 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 26 14:28:48.033660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ================================================================================ {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 26 14:28:48.033752 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] config_dir = [] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.033881 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.033964 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] admin_token = **** {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.034063 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] config_source = [] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.034153 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] config_dir = [] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.034230 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] debug = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.034341 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.034459 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.034679 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] config_source = [] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.034679 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] debug = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.034799 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.034933 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.035039 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.035143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.035221 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.035320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.035527 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.035742 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.035828 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] list_limit = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.035893 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.035952 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] log_config_append = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.036071 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.036179 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] list_limit = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.036275 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] log_dir = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.036373 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] log_config_append = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.036474 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] log_file = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.036573 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.036573 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] log_dir = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.036763 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.036876 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] log_file = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.036942 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.037004 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.037114 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.037234 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.037358 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.037482 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.037601 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.037721 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.037840 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.037961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.038070 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.038174 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.038275 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.038376 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.038477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.038583 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.038679 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.038777 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.038878 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.038973 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.039065 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.039168 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.039294 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.039440 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.039566 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.039666 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.039803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.039961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.040189 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.040293 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.97/identity {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.040476 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] publish_errors = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.040709 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.97/identity {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.040862 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.041039 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] publish_errors = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.041208 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.041208 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.041572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.041572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.041874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.042050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.042271 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.042453 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.042604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] standard_threads = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.042775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.042950 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.043128 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] standard_threads = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.043304 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.043518 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.043636 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.043636 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] use_journal = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.043829 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.043980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] use_json = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.044059 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.044184 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] use_stderr = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.044265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] use_journal = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.044341 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] use_syslog = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.044417 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] use_json = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.044499 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.044576 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] use_stderr = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.044576 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] use_syslog = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.044934 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.045041 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:28:48.045154 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.045238 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.045316 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.045388 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.045463 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.045534 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.045608 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.045689 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.045765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.045841 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.045916 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.045993 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.046068 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.046068 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.046208 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.046284 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.046359 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.046359 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.046482 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.046561 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.046633 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.046702 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.046771 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.046771 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.046905 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.046983 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.047097 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.047221 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.047296 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.047436 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.047565 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.047696 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.047818 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.047932 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.048035 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.048173 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.048284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.048990 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.049094 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.049189 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.049283 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.049451 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.049555 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.049704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.049807 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.049955 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.050060 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.050189 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.050306 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.050409 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.050508 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.050639 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.050744 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.050876 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.051011 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.051133 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.051246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.051432 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.051557 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.051671 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.051795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.051998 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.052103 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.052198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.052291 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.052384 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.052488 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.052584 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.052678 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.052777 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.052909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.053013 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.053598 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.053598 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.053731 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.053807 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.053882 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.053963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.054041 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.054119 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.054190 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.054258 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.054327 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.054404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.054476 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.054583 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.054664 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.054832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.054916 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.055067 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.055157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.055308 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.055386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.055564 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.055644 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.055804 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.057219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.057219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.057432 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.057528 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.057623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.057727 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.057831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.057928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.058022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.058124 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.058224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.058321 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.058423 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.058423 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.058625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.058707 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.058824 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.058900 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.059023 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.059100 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.059284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.059360 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.059504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.059583 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.059713 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.059795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.059868 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.059992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.060075 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.060147 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.060287 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.060372 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.060453 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.060526 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.060592 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.060769 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.060987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.061066 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.061191 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.061268 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.061382 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.061457 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.061580 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.061656 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.061772 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.061884 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.061959 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.062019 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.062150 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.062273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.062358 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.062430 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.062544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.062619 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.062800 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.062875 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.063002 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.063086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.063158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.063281 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.063363 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.063500 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.063589 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.063662 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.063733 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.063808 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.063880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.063949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.064027 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.064099 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.064175 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.064251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.064412 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.064508 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.064682 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.064775 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.064913 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.065000 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.065162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.065259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.065399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.065477 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.065639 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.065718 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.065873 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.065960 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.066106 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.066185 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.066337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.066417 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.066572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.066657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.066800 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.066878 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.067034 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.067111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.067270 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.067355 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.067529 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.067608 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.067778 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.067865 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.068022 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.068113 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.068277 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.068356 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.068520 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.068603 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.068756 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] auth.external = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.068904 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.069029 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.069108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.069277 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.069355 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.069517 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] auth.external = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.069604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.069757 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.069850 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] auth.password = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.069993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] auth.token = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.070072 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.070238 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.070318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.070477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.070572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] auth.password = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.070727 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.070814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] auth.token = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.070963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.071049 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.071202 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.071288 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.071473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.071594 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.071710 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.071836 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.071962 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.072055 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] credential.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.072210 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.072289 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.072445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.072524 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.072689 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] credential.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.072777 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.072934 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.073012 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.073185 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.073409 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.073488 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.073558 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.073671 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.073751 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.073923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.074002 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.074167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.074319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.074431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.074516 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.074691 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.074769 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.074935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.075025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.075196 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.075276 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.075488 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.075568 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.075750 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.075843 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.075990 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] federation.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.076069 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.076231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.076319 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.076481 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.076572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.076728 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.076807 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] federation.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.076961 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.077041 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.077213 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.077299 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.077459 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.077538 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.077694 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.077772 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.077941 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.078031 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.078182 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.078262 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.078468 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.078547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.078710 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.078789 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.078967 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.079044 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.079210 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.079293 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.079506 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.079594 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.079746 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.079824 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.079984 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.080064 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.080235 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.080317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.080469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.080554 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.080708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.080787 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.080964 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.081050 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.081203 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.081283 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.081455 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.081574 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.081711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.081807 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.081955 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.082042 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.082196 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.082274 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.082437 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.082517 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.082674 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.082806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.082915 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.082996 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.083170 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.083257 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.083441 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.083524 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.083710 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.083789 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.087526 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.087653 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.087772 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.087887 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.088000 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.088114 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.088242 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.088361 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.088481 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.088600 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.088713 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.088826 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.088826 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.089048 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.089160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.089273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.089273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.089483 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.089483 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.089716 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.089716 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.089933 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.090048 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.090161 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.090291 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.090402 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.090534 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.090645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.090925 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.091037 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.091166 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.091275 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.091390 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.091567 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.091681 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.091794 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.091931 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.092042 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.092155 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.092267 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.092379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.092493 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.092617 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.092727 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.092843 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.092959 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.093075 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.093203 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.093316 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.093430 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.093550 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.093664 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.093781 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.094404 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.094517 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.094634 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.094750 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.094866 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.094981 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.095109 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.095223 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.095339 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.095480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.095605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.095719 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.095852 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.095971 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.096087 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.096214 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.096329 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.096445 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.096574 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.096724 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.096836 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.096952 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.097069 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.097187 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.097317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.097430 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.097546 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.097704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.097821 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.097937 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.098065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.098177 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.098292 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.098408 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.098525 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.098640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.098768 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.098884 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.099000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.099116 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.099251 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.099251 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.099618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.099732 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.099851 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.099967 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.100085 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.100085 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.100315 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.100425 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.100553 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.100671 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.100789 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.100905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.101032 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.101146 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.101261 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.101378 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.101494 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.101611 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.101810 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.101921 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.102038 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.102191 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.102324 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.102438 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.102568 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.102680 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.102797 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.102913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.103027 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.103143 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.103271 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.103384 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.103530 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.103651 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.103771 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.103889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.104029 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.104147 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.104271 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.104390 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.104510 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.104629 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] resource.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.104760 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.104877 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.104996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.105114 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.105232 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.105351 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.105482 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] resource.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.105599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.105725 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.105843 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.105962 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.106081 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.106211 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.106328 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.106446 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.106564 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.106683 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.106802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.106932 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.107049 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] role.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.107167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.107287 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] role.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.107445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.107561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.107696 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.107808 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.107926 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] role.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.108042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.108159 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] role.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.108285 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.108396 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.108513 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.108629 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.108744 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.108857 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.109022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.109153 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.109273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.109389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.109505 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.110238 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.110362 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.110480 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.110597 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.110713 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.110830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.110945 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.111074 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.111190 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.111312 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.111448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.111448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.111678 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.111814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.111929 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.112051 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.112169 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.112169 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.112392 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.112521 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.112639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.112757 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.112882 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.113000 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.113121 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.113251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.113366 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.113486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.113962 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.114098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.114222 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.114353 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.114471 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.114590 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.114709 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.114834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.114964 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.115080 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.115200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.115320 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.115490 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.115603 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.115739 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.115851 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.115967 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.116084 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.116201 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.116317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.116446 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.116564 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] token.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.116691 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.116805 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.116923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.117038 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.117160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.117273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.117389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] token.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.117503 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.117644 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.117762 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.117894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.118006 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.118129 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.118244 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.118360 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.118476 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.118605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.118717 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.118833 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.118949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.119064 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.119180 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.119308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.119449 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.119567 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.119687 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.119801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.120048 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.127613 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.127706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.127768 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.127825 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.127890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.127960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.128044 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.128119 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.connection = **** {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.128199 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.128279 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.128352 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.128423 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.128489 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.128569 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.128642 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.connection = **** {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.128721 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.128721 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.128927 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.128927 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.129043 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.129043 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.129167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.129238 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.129308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.129378 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.129438 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.129514 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.129595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.129668 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.129739 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.129809 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.129879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.129879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.130006 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.130079 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.130149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.130149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.130272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.130272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.130397 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.130397 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.130513 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.130583 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.130653 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.130723 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 26 14:28:48.130806 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.130806 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.130806 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.130806 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:28:48.130806 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 26 14:28:48.150011 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.150240 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.150354 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.150465 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.150593 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.150832 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.159525 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.159776 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.193785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.194022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.194256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.194494 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.246730 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.246892 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.298014 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.298233 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.298346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.298471 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.364854 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.365041 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.365134 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.365274 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.365361 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.365505 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.365627 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.365721 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.365799 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.365929 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.366016 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.366137 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.389333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.389571 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.389672 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.389806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.389888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.390027 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.390109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.390245 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.390327 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.390483 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.390565 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.390701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.390792 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.391515 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.402575 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.402921 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.415661 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.415760 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.415892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.415976 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.416112 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.416195 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.416328 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.416421 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.416541 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.416623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.416749 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.416830 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.416960 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.417042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:28:48.434404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.434619 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 14:28:48.434771 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.434953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.435053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 14:28:48.435157 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 14:28:48.435317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.435439 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.435544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 14:28:48.435697 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.435812 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.435953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.436071 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.436206 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.436328 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.436584 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.446518 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.446794 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.446876 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.447061 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.447157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.447339 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.447449 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.447680 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.454274 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.454578 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.458918 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.459197 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.459279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.459509 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.459594 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.459792 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.459875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.460070 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.460153 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.460326 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.460417 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.460589 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.460671 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.460851 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.460932 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.461121 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.461210 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.461388 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.461487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.461664 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.461749 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.461928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.485137 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.485341 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.486623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.486833 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.486928 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.487112 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.491542 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.491824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.491913 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.492102 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.492184 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.492386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.498701 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.498942 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.499024 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.499284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.503818 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.504013 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.504098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.504304 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.504399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.504586 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.504668 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.504867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.504950 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.505150 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.505235 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.505429 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.505510 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.505706 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.505785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.505982 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.522195 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.522475 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.524347 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.524638 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.526413 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.528575 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.529575 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.529870 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.531774 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.532067 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.534276 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.537505 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.537598 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.540880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.541180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.541465 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.541750 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.542045 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.542188 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.542326 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.545392 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.545692 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.545997 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.546301 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.546579 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.546876 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.558806 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.559093 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.559360 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.559651 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.563943 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.564244 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.564508 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.564767 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.567251 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.568850 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.569131 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.571627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.571993 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.572276 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.572569 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.572863 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.573151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.573279 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.573433 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.576288 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.576569 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.576857 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.577149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.577424 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.584625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.584907 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.585192 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.585477 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.585766 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.586044 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.586323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.588867 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.589147 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.589426 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.589706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.589986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.590260 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.590534 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.601889 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.604874 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.606072 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.607848 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.609027 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.610890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.612520 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.612662 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.612816 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.613100 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.615040 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.616589 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.616887 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.617167 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.619588 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.621186 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.623612 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.624166 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.624458 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.624745 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.625020 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.625816 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.628784 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.629075 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.629360 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.629634 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.633601 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.633892 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.634174 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.637633 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.637923 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.638202 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.641578 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.641857 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.642136 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.642412 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.642695 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.642976 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.643248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.643558 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.643935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.644218 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.644512 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.645041 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.645315 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.645577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.645913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:28:48.646194 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.646472 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.646747 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.647028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.647307 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.647605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.647891 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.648261 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.648530 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.648816 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.649087 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.649369 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.649629 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:28:48.679494 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55a77c2b49c0 pid: 80523 (default app) Jul 26 14:28:48.682748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1dd54-8b62-464a-9ee0-c5f2cdad16b9 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:48.683039 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1dd54-8b62-464a-9ee0-c5f2cdad16b9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:48.683316 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1dd54-8b62-464a-9ee0-c5f2cdad16b9 None None] PATH_INFO: `/v3` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:48.683417 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55a77c2b49c0 pid: 80522 (default app) Jul 26 14:28:48.687695 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1/1] 10.222.0.97 () {50 vars in 791 bytes} [Tue Jul 26 14:28:48 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:28:57.526005 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac54abf-35df-43bd-867d-1edbab0e3cad None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:57.526495 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac54abf-35df-43bd-867d-1edbab0e3cad None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:57.526598 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac54abf-35df-43bd-867d-1edbab0e3cad None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:57.531614 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1/2] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:28:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:28:57.536662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69d290-5509-4bfe-8d4a-c0d01f3d786c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:57.537046 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69d290-5509-4bfe-8d4a-c0d01f3d786c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:57.537213 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69d290-5509-4bfe-8d4a-c0d01f3d786c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:57.607149 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: INFO dbcounter [None req-8a69d290-5509-4bfe-8d4a-c0d01f3d786c None None] Registered counter for database keystone Jul 26 14:28:57.627953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG oslo_db.sqlalchemy.engines [None req-8a69d290-5509-4bfe-8d4a-c0d01f3d786c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80523) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 26 14:28:57.634062 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writer thread running {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 26 14:28:57.817452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG passlib.handlers.bcrypt [None req-8a69d290-5509-4bfe-8d4a-c0d01f3d786c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80523) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 26 14:28:57.817788 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG passlib.handlers.bcrypt [None req-8a69d290-5509-4bfe-8d4a-c0d01f3d786c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80523) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 26 14:28:57.872455 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-8a69d290-5509-4bfe-8d4a-c0d01f3d786c None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:28:57.874103 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8a69d290-5509-4bfe-8d4a-c0d01f3d786c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:28:58.002727 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 2/3] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:28:57 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 26 14:28:58.160281 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac54abf-35df-43bd-867d-1edbab0e3cad None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:28:58.231348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: INFO dbcounter [None req-8ac54abf-35df-43bd-867d-1edbab0e3cad None None] Registered counter for database keystone Jul 26 14:28:58.252006 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG oslo_db.sqlalchemy.engines [None req-8ac54abf-35df-43bd-867d-1edbab0e3cad None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80522) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 26 14:28:58.258166 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writer thread running {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 26 14:28:58.376805 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac54abf-35df-43bd-867d-1edbab0e3cad None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:28:58.381153 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3cd0d-c4dd-4388-8bdc-2c3087b024f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:28:58.381797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3cd0d-c4dd-4388-8bdc-2c3087b024f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:58.382085 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3cd0d-c4dd-4388-8bdc-2c3087b024f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:58.382370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3cd0d-c4dd-4388-8bdc-2c3087b024f8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:58.390664 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be3cd0d-c4dd-4388-8bdc-2c3087b024f8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:28:58.442589 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8be3cd0d-c4dd-4388-8bdc-2c3087b024f8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 26 14:28:58.442882 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be3cd0d-c4dd-4388-8bdc-2c3087b024f8 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:28:58.452409 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 2/4] 10.222.0.97 () {58 vars in 1324 bytes} [Tue Jul 26 14:28:58 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 14:28:58.458368 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a69d290-5509-4bfe-8d4a-c0d01f3d786c None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:28:58.472665 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a69d290-5509-4bfe-8d4a-c0d01f3d786c None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:28:58.477136 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d37c3-2a79-4203-88a5-87e6eec70fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:28:58.478248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d37c3-2a79-4203-88a5-87e6eec70fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:58.478539 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d37c3-2a79-4203-88a5-87e6eec70fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:58.478822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d37c3-2a79-4203-88a5-87e6eec70fc0 admin admin] PATH_INFO: `/v3/projects/011581630bbe4c60a042395d12032595` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:58.480362 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d37c3-2a79-4203-88a5-87e6eec70fc0 admin admin] RBAC: Authorizing `identity:get_project(project_id=011581630bbe4c60a042395d12032595)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:28:58.530323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7d6d37c3-2a79-4203-88a5-87e6eec70fc0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 26 14:28:58.530681 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d37c3-2a79-4203-88a5-87e6eec70fc0 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:28:58.535867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 3/5] 10.222.0.97 () {56 vars in 1207 bytes} [Tue Jul 26 14:28:58 2022] GET /identity/v3/projects/011581630bbe4c60a042395d12032595 => generated 344 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:00.513913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5903f-7a80-47ea-9149-f9c8be04d895 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:00.514408 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5903f-7a80-47ea-9149-f9c8be04d895 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:00.514493 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5903f-7a80-47ea-9149-f9c8be04d895 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:00.518596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 3/6] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:29:00.524226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18807c-b436-4b18-9ba6-8c8d23bac096 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:00.524631 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18807c-b436-4b18-9ba6-8c8d23bac096 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:00.524794 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18807c-b436-4b18-9ba6-8c8d23bac096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:00.544581 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-db18807c-b436-4b18-9ba6-8c8d23bac096 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:00.546110 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-db18807c-b436-4b18-9ba6-8c8d23bac096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:00.554549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 4/7] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:00 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 26 14:29:00.576505 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5903f-7a80-47ea-9149-f9c8be04d895 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:00.588581 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5903f-7a80-47ea-9149-f9c8be04d895 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:00.592758 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77a097-e0c1-47e3-85c1-55a423b3d497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:00.593362 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77a097-e0c1-47e3-85c1-55a423b3d497 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:00.593650 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77a097-e0c1-47e3-85c1-55a423b3d497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:00.593931 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77a097-e0c1-47e3-85c1-55a423b3d497 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:00.601506 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77a097-e0c1-47e3-85c1-55a423b3d497 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:00.603284 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77a097-e0c1-47e3-85c1-55a423b3d497 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:00.613463 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 4/8] 10.222.0.97 () {58 vars in 1324 bytes} [Tue Jul 26 14:29:00 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 14:29:00.618662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18807c-b436-4b18-9ba6-8c8d23bac096 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:00.628147 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18807c-b436-4b18-9ba6-8c8d23bac096 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:00.632549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9b64f-c289-43f9-a5c6-48f58499de2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:00.633182 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9b64f-c289-43f9-a5c6-48f58499de2b admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:00.633470 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9b64f-c289-43f9-a5c6-48f58499de2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:00.633753 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9b64f-c289-43f9-a5c6-48f58499de2b admin admin] PATH_INFO: `/v3/users/11071bdab076477385007310e6ab9852` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:00.635118 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9b64f-c289-43f9-a5c6-48f58499de2b admin admin] RBAC: Authorizing `identity:get_user(user_id=11071bdab076477385007310e6ab9852)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:00.636991 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9b64f-c289-43f9-a5c6-48f58499de2b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:00.638541 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 5/9] 10.222.0.97 () {56 vars in 1198 bytes} [Tue Jul 26 14:29:00 2022] GET /identity/v3/users/11071bdab076477385007310e6ab9852 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:02.701389 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c076b-51bb-426b-b50e-58a11fe5469a None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:02.701905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c076b-51bb-426b-b50e-58a11fe5469a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:02.701992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c076b-51bb-426b-b50e-58a11fe5469a None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:02.703316 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 5/10] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:02.708515 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb4f77-d3d9-44e2-9597-e0d186e6b410 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:02.708923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb4f77-d3d9-44e2-9597-e0d186e6b410 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:02.709076 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb4f77-d3d9-44e2-9597-e0d186e6b410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:02.719718 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf39953-4237-4db7-8eac-92a6e339590a None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:02.720004 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf39953-4237-4db7-8eac-92a6e339590a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:02.720279 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf39953-4237-4db7-8eac-92a6e339590a None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:02.721587 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 6/11] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:29:02.726557 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db6793-9b7b-45ad-b520-b5ad2afeb79a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:02.726859 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db6793-9b7b-45ad-b520-b5ad2afeb79a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:02.727136 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db6793-9b7b-45ad-b520-b5ad2afeb79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:02.728284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-d2fb4f77-d3d9-44e2-9597-e0d186e6b410 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:02.729493 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d2fb4f77-d3d9-44e2-9597-e0d186e6b410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:02.769444 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG passlib.handlers.bcrypt [None req-71db6793-9b7b-45ad-b520-b5ad2afeb79a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80522) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 26 14:29:02.769877 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG passlib.handlers.bcrypt [None req-71db6793-9b7b-45ad-b520-b5ad2afeb79a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80522) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 26 14:29:02.809411 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 6/12] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:02.815528 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-71db6793-9b7b-45ad-b520-b5ad2afeb79a None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:02.817136 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-71db6793-9b7b-45ad-b520-b5ad2afeb79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:02.832467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb4f77-d3d9-44e2-9597-e0d186e6b410 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:02.857484 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:02.921848 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 7/13] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:29:02.941713 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb4f77-d3d9-44e2-9597-e0d186e6b410 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:02.948154 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db6793-9b7b-45ad-b520-b5ad2afeb79a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:02.981077 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993d878-18e9-4dfe-aae3-f1673e3e57d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:02.981653 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993d878-18e9-4dfe-aae3-f1673e3e57d7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:02.981942 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993d878-18e9-4dfe-aae3-f1673e3e57d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:02.982226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993d878-18e9-4dfe-aae3-f1673e3e57d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:02.993940 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993d878-18e9-4dfe-aae3-f1673e3e57d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:03.002448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-c993d878-18e9-4dfe-aae3-f1673e3e57d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:29:03.002448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:03.002448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:03.002448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:03.002448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:03.002448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:03.002448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:03.002448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:03.002448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:03.002448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:03.002448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:03.002448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:03.002448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:03.002448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:03.002448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:03.002448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:03.003277 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:03.003277 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:03.003277 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:03.003277 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:03.003277 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:03.003277 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:03.003277 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:03.003277 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:03.003277 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:03.003277 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:03.003277 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:03.003277 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:03.003277 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:03.003277 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:03.003277 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:03.003277 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:03.004355 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:03.004355 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:03.004355 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:03.004355 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:03.004355 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:03.004355 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:03.004355 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:03.004355 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:03.004355 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:03.004355 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:03.004355 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:29:03.004355 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:29:03.004355 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 7/14] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:29:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:29:03.009558 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993d878-18e9-4dfe-aae3-f1673e3e57d7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:03.051728 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db6793-9b7b-45ad-b520-b5ad2afeb79a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:03.054086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:03.089513 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773fe044-07d9-4633-8090-6e004c253966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:03.090402 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773fe044-07d9-4633-8090-6e004c253966 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:03.090688 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773fe044-07d9-4633-8090-6e004c253966 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:03.090964 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773fe044-07d9-4633-8090-6e004c253966 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:03.094534 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:03.111899 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993d878-18e9-4dfe-aae3-f1673e3e57d7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:03.146239 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0364b-14e5-4d6d-b4e0-c0e1fb643fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:03.146878 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0364b-14e5-4d6d-b4e0-c0e1fb643fbd None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:03.147159 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0364b-14e5-4d6d-b4e0-c0e1fb643fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:03.147461 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0364b-14e5-4d6d-b4e0-c0e1fb643fbd None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:03.148178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb0364b-14e5-4d6d-b4e0-c0e1fb643fbd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:03.148484 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb0364b-14e5-4d6d-b4e0-c0e1fb643fbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:03.149885 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb0364b-14e5-4d6d-b4e0-c0e1fb643fbd None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:03.158396 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773fe044-07d9-4633-8090-6e004c253966 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:03.159949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773fe044-07d9-4633-8090-6e004c253966 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:03.162567 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 8/15] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:03.167997 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0364b-14e5-4d6d-b4e0-c0e1fb643fbd None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:03.258204 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 8/16] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:29:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 14:29:03.263847 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773fe044-07d9-4633-8090-6e004c253966 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:03.267716 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:03.268246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0364b-14e5-4d6d-b4e0-c0e1fb643fbd None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:03.303420 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de4549-c999-4d0b-bda6-a22edf47936e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:03.304067 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de4549-c999-4d0b-bda6-a22edf47936e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:03.304353 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de4549-c999-4d0b-bda6-a22edf47936e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:03.304631 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de4549-c999-4d0b-bda6-a22edf47936e None admin] PATH_INFO: `/v3/users/11071bdab076477385007310e6ab9852` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:03.305815 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de4549-c999-4d0b-bda6-a22edf47936e None admin] RBAC: Authorizing `identity:get_user(user_id=11071bdab076477385007310e6ab9852)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:03.307267 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de4549-c999-4d0b-bda6-a22edf47936e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:03.308872 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 9/17] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:03 2022] GET /identity/v3/users/11071bdab076477385007310e6ab9852 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:03.314881 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de4549-c999-4d0b-bda6-a22edf47936e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:03.330077 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:03.362671 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773fe044-07d9-4633-8090-6e004c253966 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:03.402332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5535e01-4176-4d54-9b79-948839d27421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:03.403222 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5535e01-4176-4d54-9b79-948839d27421 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:03.403569 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5535e01-4176-4d54-9b79-948839d27421 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:03.403860 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5535e01-4176-4d54-9b79-948839d27421 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:03.427579 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5535e01-4176-4d54-9b79-948839d27421 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:03.429073 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5535e01-4176-4d54-9b79-948839d27421 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:03.448623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-b5535e01-4176-4d54-9b79-948839d27421 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:29:03.448623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:03.448623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:03.448623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:03.448623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:03.448623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:03.448623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:03.448623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:03.448623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:03.448623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:03.448623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:03.448623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:03.448623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:29:03.448623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:29:03.448623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:29:03.449641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:29:03.449641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:03.449641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:03.449641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:03.449641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:03.449641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:03.449641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:03.449641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:03.449641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:03.449641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:03.449641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:03.449641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:03.449641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:03.449641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:03.449641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:03.449641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:03.449641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:03.450471 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:29:03.450471 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:29:03.450471 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:29:03.450471 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:29:03.450471 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 9/18] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:29:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:29:03.457352 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5535e01-4176-4d54-9b79-948839d27421 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:03.512938 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:03.521677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de4549-c999-4d0b-bda6-a22edf47936e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:03.556701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f3a5c-e06d-4f10-81b6-4d324bbf673a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:03.557352 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f3a5c-e06d-4f10-81b6-4d324bbf673a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:03.557633 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f3a5c-e06d-4f10-81b6-4d324bbf673a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:03.557910 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f3a5c-e06d-4f10-81b6-4d324bbf673a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:03.559076 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612f3a5c-e06d-4f10-81b6-4d324bbf673a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:03.560011 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5535e01-4176-4d54-9b79-948839d27421 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:03.560575 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612f3a5c-e06d-4f10-81b6-4d324bbf673a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:03.562186 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 10/19] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:29:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:03.567840 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f3a5c-e06d-4f10-81b6-4d324bbf673a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:03.593452 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd9b4d-c179-40c0-8fa9-c3ce96a0ffc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:03.594070 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd9b4d-c179-40c0-8fa9-c3ce96a0ffc7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:03.594355 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd9b4d-c179-40c0-8fa9-c3ce96a0ffc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:03.594630 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd9b4d-c179-40c0-8fa9-c3ce96a0ffc7 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:03.595328 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd9b4d-c179-40c0-8fa9-c3ce96a0ffc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:03.595668 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd9b4d-c179-40c0-8fa9-c3ce96a0ffc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:03.597045 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd9b4d-c179-40c0-8fa9-c3ce96a0ffc7 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:03.614428 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 10/20] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:03.651497 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:03.672201 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f3a5c-e06d-4f10-81b6-4d324bbf673a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:03.709177 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9c9af-7404-4dbf-8a3f-dbaa1d377e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:03.709831 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9c9af-7404-4dbf-8a3f-dbaa1d377e8a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:03.710086 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9c9af-7404-4dbf-8a3f-dbaa1d377e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:03.710365 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9c9af-7404-4dbf-8a3f-dbaa1d377e8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:03.711170 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9c9af-7404-4dbf-8a3f-dbaa1d377e8a None admin] RBAC: Adding query filter params (role.id=51b2d10dfffb4b75b186f3d8b15e2a78, scope.domain.id=default, user.id=11071bdab076477385007310e6ab9852) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:03.711513 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9c9af-7404-4dbf-8a3f-dbaa1d377e8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:03.712966 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9c9af-7404-4dbf-8a3f-dbaa1d377e8a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:03.720582 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 11/21] 10.222.0.97 () {56 vars in 1322 bytes} [Tue Jul 26 14:29:03 2022] GET /identity/v3/role_assignments?user.id=11071bdab076477385007310e6ab9852&scope.domain.id=default&role.id=51b2d10dfffb4b75b186f3d8b15e2a78 => generated 233 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:06.061419 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc8672-b26e-4a6e-a134-91214e6344ab None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:06.062422 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc8672-b26e-4a6e-a134-91214e6344ab None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:06.062496 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc8672-b26e-4a6e-a134-91214e6344ab None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:06.064160 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 11/22] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:06.069151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14ab98-5e9d-46a0-af89-3129c468d66c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:06.069554 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14ab98-5e9d-46a0-af89-3129c468d66c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:06.069745 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14ab98-5e9d-46a0-af89-3129c468d66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:06.096393 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-fc14ab98-5e9d-46a0-af89-3129c468d66c None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:06.097610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-fc14ab98-5e9d-46a0-af89-3129c468d66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:06.189309 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 12/23] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:06.229231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc8672-b26e-4a6e-a134-91214e6344ab None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:06.242547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2fb26-ecc4-4516-8036-c2f9e13688b7 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:06.243152 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2fb26-ecc4-4516-8036-c2f9e13688b7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:06.243653 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2fb26-ecc4-4516-8036-c2f9e13688b7 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:06.245202 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 13/24] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:06.251417 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efdac4-068d-4ced-96ce-85f194751ab4 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:06.252092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efdac4-068d-4ced-96ce-85f194751ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:06.254023 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efdac4-068d-4ced-96ce-85f194751ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:06.293435 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b8efdac4-068d-4ced-96ce-85f194751ab4 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:06.296389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b8efdac4-068d-4ced-96ce-85f194751ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:06.425292 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 14/25] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:29:06.426055 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc8672-b26e-4a6e-a134-91214e6344ab None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:06.426703 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b19718-8bab-41ec-9674-867cbb05e837 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:06.427001 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b19718-8bab-41ec-9674-867cbb05e837 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:06.427284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b19718-8bab-41ec-9674-867cbb05e837 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:06.429744 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 15/26] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:06.434756 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154421d-97c9-4f71-ae24-b90dc9bac26a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:06.434756 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154421d-97c9-4f71-ae24-b90dc9bac26a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:06.435046 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154421d-97c9-4f71-ae24-b90dc9bac26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:06.445025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:06.454669 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:06.472014 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b154421d-97c9-4f71-ae24-b90dc9bac26a None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:06.473426 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b154421d-97c9-4f71-ae24-b90dc9bac26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:06.476022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468343d-a196-402d-b714-8ae98cebc90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:06.477300 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468343d-a196-402d-b714-8ae98cebc90f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:06.477300 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468343d-a196-402d-b714-8ae98cebc90f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:06.477300 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468343d-a196-402d-b714-8ae98cebc90f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:06.488823 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2468343d-a196-402d-b714-8ae98cebc90f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:06.490465 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2468343d-a196-402d-b714-8ae98cebc90f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:06.505710 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-2468343d-a196-402d-b714-8ae98cebc90f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:29:06.505710 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:06.505710 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:06.505710 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:06.505710 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:06.505710 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:06.505710 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:06.505710 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:06.505710 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:06.505710 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:06.505710 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:06.505710 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:06.505710 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:29:06.505710 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:29:06.505710 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:29:06.506704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:29:06.506704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:06.506704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:06.506704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:06.506704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:06.506704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:06.506704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:06.506704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:06.506704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:06.506704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:06.506704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:06.506704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:06.506704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:06.506704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:06.506704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:06.506704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:06.506704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:06.507893 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:29:06.507893 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:29:06.507893 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:29:06.507893 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:29:06.507893 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 12/27] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:29:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:06.507893 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468343d-a196-402d-b714-8ae98cebc90f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:06.587344 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 16/28] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:29:06.588336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154421d-97c9-4f71-ae24-b90dc9bac26a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:06.629301 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468343d-a196-402d-b714-8ae98cebc90f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:06.668481 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e902c0-9727-4253-bb94-656ab84ab719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:06.669196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e902c0-9727-4253-bb94-656ab84ab719 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:06.669484 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e902c0-9727-4253-bb94-656ab84ab719 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:06.669762 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e902c0-9727-4253-bb94-656ab84ab719 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:06.682879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e902c0-9727-4253-bb94-656ab84ab719 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:06.695195 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154421d-97c9-4f71-ae24-b90dc9bac26a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:06.739240 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a4e62-48f5-4361-bc76-e8d83a19e764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:06.740679 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a4e62-48f5-4361-bc76-e8d83a19e764 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:06.740679 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a4e62-48f5-4361-bc76-e8d83a19e764 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:06.740679 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a4e62-48f5-4361-bc76-e8d83a19e764 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:06.741225 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a4e62-48f5-4361-bc76-e8d83a19e764 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:06.741540 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a4e62-48f5-4361-bc76-e8d83a19e764 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:06.742991 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a4e62-48f5-4361-bc76-e8d83a19e764 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:06.745442 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:06.764474 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 17/29] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:29:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:06.765911 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04db415-91f8-40f2-a87d-b7e41b9a604b None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:06.766210 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04db415-91f8-40f2-a87d-b7e41b9a604b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:06.766492 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04db415-91f8-40f2-a87d-b7e41b9a604b None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:06.767826 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 18/30] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:06.768532 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04db415-91f8-40f2-a87d-b7e41b9a604b None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:06.814960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-37e902c0-9727-4253-bb94-656ab84ab719 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:29:06.814960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:06.814960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:06.814960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:06.814960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:06.814960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:06.814960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:06.814960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:06.814960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:06.814960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:06.814960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:06.814960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:06.814960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:06.814960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:06.814960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:06.814960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:06.816083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:06.816083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:06.816083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:06.816083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:06.816083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:06.816083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:06.816083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:06.816083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:06.816083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:06.816083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:06.816083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:06.816083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:06.816083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:06.816083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:06.816083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:06.816083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:06.817044 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:06.817044 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:06.817044 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:06.817044 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:06.817044 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:06.817044 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:06.817044 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:06.817044 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:06.817044 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:06.817044 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:06.817044 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:29:06.817044 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:29:06.818993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 13/31] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:29:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:06.820130 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e902c0-9727-4253-bb94-656ab84ab719 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:06.852920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:06.886528 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04db415-91f8-40f2-a87d-b7e41b9a604b None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:06.923666 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cde703-c4f0-4cc8-8d11-a2d20ddd8c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:06.924297 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cde703-c4f0-4cc8-8d11-a2d20ddd8c63 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:06.924585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cde703-c4f0-4cc8-8d11-a2d20ddd8c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:06.924873 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cde703-c4f0-4cc8-8d11-a2d20ddd8c63 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:06.925782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cde703-c4f0-4cc8-8d11-a2d20ddd8c63 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:06.927287 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cde703-c4f0-4cc8-8d11-a2d20ddd8c63 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:06.937017 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e902c0-9727-4253-bb94-656ab84ab719 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:06.973277 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40925cc9-8475-4736-a686-7245d2655d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:06.973904 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40925cc9-8475-4736-a686-7245d2655d08 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:06.974203 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40925cc9-8475-4736-a686-7245d2655d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:06.974477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40925cc9-8475-4736-a686-7245d2655d08 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:06.975458 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40925cc9-8475-4736-a686-7245d2655d08 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:06.976906 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40925cc9-8475-4736-a686-7245d2655d08 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:07.106994 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 19/32] 10.222.0.97 () {60 vars in 1131 bytes} [Tue Jul 26 14:29:06 2022] POST /identity/v3/roles => generated 225 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:29:07.108667 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1e8e5-c5f9-485a-beda-5b36126ec492 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:07.108963 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1e8e5-c5f9-485a-beda-5b36126ec492 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:07.109248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1e8e5-c5f9-485a-beda-5b36126ec492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:07.131133 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-bfa1e8e5-c5f9-485a-beda-5b36126ec492 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:07.132730 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-bfa1e8e5-c5f9-485a-beda-5b36126ec492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:07.237171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 14/33] 10.222.0.97 () {60 vars in 1141 bytes} [Tue Jul 26 14:29:06 2022] POST /identity/v3/projects => generated 301 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:29:07.239058 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40925cc9-8475-4736-a686-7245d2655d08 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:07.256529 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:07.276430 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 20/34] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:07.277990 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529cf74-9476-4a90-b7eb-3d454c7595c9 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:07.284830 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529cf74-9476-4a90-b7eb-3d454c7595c9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:07.284830 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529cf74-9476-4a90-b7eb-3d454c7595c9 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:07.284830 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 21/35] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:07.286324 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c41bd-c9a2-47dd-b064-cb55c1ffcc6b None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:07.286575 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c41bd-c9a2-47dd-b064-cb55c1ffcc6b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:07.286867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c41bd-c9a2-47dd-b064-cb55c1ffcc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:07.309297 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e33c41bd-c9a2-47dd-b064-cb55c1ffcc6b None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:07.311045 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e33c41bd-c9a2-47dd-b064-cb55c1ffcc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:07.359135 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:07.374222 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40925cc9-8475-4736-a686-7245d2655d08 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:07.402913 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 22/36] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:29:07.404297 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c41bd-c9a2-47dd-b064-cb55c1ffcc6b None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:07.428277 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68df2d-d444-4750-83e0-e01cfd383623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:07.428945 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68df2d-d444-4750-83e0-e01cfd383623 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:07.429245 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68df2d-d444-4750-83e0-e01cfd383623 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:07.433738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68df2d-d444-4750-83e0-e01cfd383623 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:07.433738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68df2d-d444-4750-83e0-e01cfd383623 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:07.433738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68df2d-d444-4750-83e0-e01cfd383623 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:07.433738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68df2d-d444-4750-83e0-e01cfd383623 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:07.445225 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 15/37] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:07.446118 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68df2d-d444-4750-83e0-e01cfd383623 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:07.523310 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c41bd-c9a2-47dd-b064-cb55c1ffcc6b None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:07.541945 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:07.561002 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0830b36-ab09-4ff3-8e70-90fa40486891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:07.562222 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0830b36-ab09-4ff3-8e70-90fa40486891 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:07.562432 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0830b36-ab09-4ff3-8e70-90fa40486891 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:07.562711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0830b36-ab09-4ff3-8e70-90fa40486891 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:07.563451 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0830b36-ab09-4ff3-8e70-90fa40486891 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:07.564970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0830b36-ab09-4ff3-8e70-90fa40486891 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:07.568059 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68df2d-d444-4750-83e0-e01cfd383623 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:07.603354 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f51eb-02bb-4d73-91bd-f4852c3687eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:07.604038 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f51eb-02bb-4d73-91bd-f4852c3687eb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:07.604324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f51eb-02bb-4d73-91bd-f4852c3687eb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:07.604605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f51eb-02bb-4d73-91bd-f4852c3687eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:07.605868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1f51eb-02bb-4d73-91bd-f4852c3687eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:07.607320 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1f51eb-02bb-4d73-91bd-f4852c3687eb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:07.609519 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 16/38] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:29:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:07.610278 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f51eb-02bb-4d73-91bd-f4852c3687eb None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:07.626214 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:07.668432 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 23/39] 10.222.0.97 () {60 vars in 1131 bytes} [Tue Jul 26 14:29:07 2022] POST /identity/v3/roles => generated 219 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:29:07.669156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0830b36-ab09-4ff3-8e70-90fa40486891 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:07.721989 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f51eb-02bb-4d73-91bd-f4852c3687eb None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:07.760980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1198b4-fedb-4125-8e8a-98a77868c8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:07.761748 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1198b4-fedb-4125-8e8a-98a77868c8ff None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:07.762113 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1198b4-fedb-4125-8e8a-98a77868c8ff None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:07.762475 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1198b4-fedb-4125-8e8a-98a77868c8ff None admin] PATH_INFO: `/v3/users/11071bdab076477385007310e6ab9852` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:07.764121 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1198b4-fedb-4125-8e8a-98a77868c8ff None admin] RBAC: Authorizing `identity:get_user(user_id=11071bdab076477385007310e6ab9852)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:07.765645 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1198b4-fedb-4125-8e8a-98a77868c8ff None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:07.772204 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 17/40] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:07 2022] GET /identity/v3/users/11071bdab076477385007310e6ab9852 => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:07.772204 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d091536-edb7-4dd1-b4f4-28c880a61b3c None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:07.772204 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d091536-edb7-4dd1-b4f4-28c880a61b3c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:07.772204 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d091536-edb7-4dd1-b4f4-28c880a61b3c None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:07.772204 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 18/41] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:07.773113 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d091536-edb7-4dd1-b4f4-28c880a61b3c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:07.788476 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0830b36-ab09-4ff3-8e70-90fa40486891 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:07.826709 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3b9d4-2ddd-4d0b-930d-8c3c4b0e2c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:07.827475 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3b9d4-2ddd-4d0b-930d-8c3c4b0e2c5c None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:07.827830 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3b9d4-2ddd-4d0b-930d-8c3c4b0e2c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:07.828206 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3b9d4-2ddd-4d0b-930d-8c3c4b0e2c5c None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:07.828901 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3b9d4-2ddd-4d0b-930d-8c3c4b0e2c5c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:07.830362 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3b9d4-2ddd-4d0b-930d-8c3c4b0e2c5c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:07.868152 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:07.891160 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d091536-edb7-4dd1-b4f4-28c880a61b3c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:07.910060 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 24/42] 10.222.0.97 () {60 vars in 1141 bytes} [Tue Jul 26 14:29:07 2022] POST /identity/v3/projects => generated 312 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:29:07.911383 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53002566-a92a-453f-95aa-768404691653 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:07.911791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53002566-a92a-453f-95aa-768404691653 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:07.912190 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53002566-a92a-453f-95aa-768404691653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:07.927037 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc0b855-3fb9-4d00-888d-1a8039979922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:07.927766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0b855-3fb9-4d00-888d-1a8039979922 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:07.928049 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0b855-3fb9-4d00-888d-1a8039979922 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:07.928333 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0b855-3fb9-4d00-888d-1a8039979922 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:07.929574 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc0b855-3fb9-4d00-888d-1a8039979922 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:07.931082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc0b855-3fb9-4d00-888d-1a8039979922 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:07.932941 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 19/43] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:29:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:07.935685 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-53002566-a92a-453f-95aa-768404691653 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:07.937161 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-53002566-a92a-453f-95aa-768404691653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:07.940360 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc0b855-3fb9-4d00-888d-1a8039979922 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:07.961652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:08.017413 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 25/44] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:08.039528 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53002566-a92a-453f-95aa-768404691653 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:08.061421 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc0b855-3fb9-4d00-888d-1a8039979922 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:08.100834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e7898-4304-40ed-ad57-4bf7b1d93ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:08.101522 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e7898-4304-40ed-ad57-4bf7b1d93ab2 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:08.101811 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e7898-4304-40ed-ad57-4bf7b1d93ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:08.102097 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e7898-4304-40ed-ad57-4bf7b1d93ab2 None admin] PATH_INFO: `/v3/domains/default/users/11071bdab076477385007310e6ab9852/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:08.104141 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e7898-4304-40ed-ad57-4bf7b1d93ab2 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=11071bdab076477385007310e6ab9852, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:08.105607 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e7898-4304-40ed-ad57-4bf7b1d93ab2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:08.153360 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53002566-a92a-453f-95aa-768404691653 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:08.187918 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2cb31b-f139-463d-bfdf-b7a55b2d7149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:08.188613 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2cb31b-f139-463d-bfdf-b7a55b2d7149 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:08.188904 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2cb31b-f139-463d-bfdf-b7a55b2d7149 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:08.189187 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2cb31b-f139-463d-bfdf-b7a55b2d7149 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:08.189967 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2cb31b-f139-463d-bfdf-b7a55b2d7149 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:08.191431 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2cb31b-f139-463d-bfdf-b7a55b2d7149 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:08.194352 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 20/45] 10.222.0.97 () {58 vars in 1361 bytes} [Tue Jul 26 14:29:07 2022] PUT /identity/v3/domains/default/users/11071bdab076477385007310e6ab9852/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:29:08.286686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 26/46] 10.222.0.97 () {60 vars in 1131 bytes} [Tue Jul 26 14:29:08 2022] POST /identity/v3/roles => generated 223 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:29:09.916995 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e292a5-fae0-4845-b4be-b45eb16b054f None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:10.187003 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e292a5-fae0-4845-b4be-b45eb16b054f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:10.187003 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e292a5-fae0-4845-b4be-b45eb16b054f None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:10.187003 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 21/47] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:10.187855 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8a58a-f76e-46fe-ab2a-80a630fca9e6 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:10.187855 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8a58a-f76e-46fe-ab2a-80a630fca9e6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:10.187855 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8a58a-f76e-46fe-ab2a-80a630fca9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:10.187855 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-31a8a58a-f76e-46fe-ab2a-80a630fca9e6 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:10.187855 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-31a8a58a-f76e-46fe-ab2a-80a630fca9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:10.187855 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:10.188561 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 27/48] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:10.188638 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e292a5-fae0-4845-b4be-b45eb16b054f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:10.188638 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:10.188638 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e292a5-fae0-4845-b4be-b45eb16b054f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:10.188638 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55257f7-f485-4a66-8b8e-96f05a424d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:10.188944 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55257f7-f485-4a66-8b8e-96f05a424d59 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:10.188944 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55257f7-f485-4a66-8b8e-96f05a424d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:10.188944 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55257f7-f485-4a66-8b8e-96f05a424d59 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:10.188944 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55257f7-f485-4a66-8b8e-96f05a424d59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:10.188944 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55257f7-f485-4a66-8b8e-96f05a424d59 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:10.188944 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 22/49] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:29:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:10.189358 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8a58a-f76e-46fe-ab2a-80a630fca9e6 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:10.189428 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbdb4d-efad-4a28-8f31-127ab4fb762a None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:10.189428 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbdb4d-efad-4a28-8f31-127ab4fb762a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:10.189428 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbdb4d-efad-4a28-8f31-127ab4fb762a None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:10.189428 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 23/50] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:29:10.190552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badabc04-78ca-4e73-9e0b-5f01640d47b4 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:10.190846 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badabc04-78ca-4e73-9e0b-5f01640d47b4 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:10.191131 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badabc04-78ca-4e73-9e0b-5f01640d47b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:10.210281 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-badabc04-78ca-4e73-9e0b-5f01640d47b4 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:10.211952 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-badabc04-78ca-4e73-9e0b-5f01640d47b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:10.278355 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 24/51] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:29:10.284139 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8a58a-f76e-46fe-ab2a-80a630fca9e6 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:10.301395 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badabc04-78ca-4e73-9e0b-5f01640d47b4 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:10.304643 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:10.317629 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2b727-09c3-42b3-93b3-e3952a0956f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:10.322607 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2b727-09c3-42b3-93b3-e3952a0956f0 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:10.322607 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2b727-09c3-42b3-93b3-e3952a0956f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:10.322607 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2b727-09c3-42b3-93b3-e3952a0956f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:10.322607 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2b727-09c3-42b3-93b3-e3952a0956f0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:10.322607 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2b727-09c3-42b3-93b3-e3952a0956f0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:10.388483 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:10.395625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 28/52] 10.222.0.97 () {60 vars in 1140 bytes} [Tue Jul 26 14:29:10 2022] POST /identity/v3/projects => generated 298 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:29:10.397761 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badabc04-78ca-4e73-9e0b-5f01640d47b4 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:10.435262 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b457ad8c-3cd9-40e1-b88c-c5a7aed9a496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:10.435958 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457ad8c-3cd9-40e1-b88c-c5a7aed9a496 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:10.436247 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457ad8c-3cd9-40e1-b88c-c5a7aed9a496 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:10.436528 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457ad8c-3cd9-40e1-b88c-c5a7aed9a496 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:10.448823 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b457ad8c-3cd9-40e1-b88c-c5a7aed9a496 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:10.454178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-b457ad8c-3cd9-40e1-b88c-c5a7aed9a496 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:29:10.454178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:10.454178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:10.454178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:10.454178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:10.454178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:10.454178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:10.454178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:10.454178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:10.454178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:10.454178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:10.454178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:10.454178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:10.454178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:10.454178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:10.454178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:10.455200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:10.455200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:10.455200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:10.455200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:10.455200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:10.455200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:10.455200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:10.455200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:10.455200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:10.455200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:10.455200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:10.455200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:10.455200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:10.455200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:10.455200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:10.455200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:10.456018 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:10.456018 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:10.456018 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:10.456018 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:10.456018 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:10.456018 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:10.456018 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:10.456018 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:10.456018 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:10.456018 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:10.456018 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:29:10.456018 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:29:10.456018 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 25/53] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:29:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:29:10.462999 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2b727-09c3-42b3-93b3-e3952a0956f0 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:10.568793 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2b727-09c3-42b3-93b3-e3952a0956f0 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:10.602397 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2c98a-f8f5-493d-8e09-e400acf6a7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:10.603029 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2c98a-f8f5-493d-8e09-e400acf6a7ba None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:10.603317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2c98a-f8f5-493d-8e09-e400acf6a7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:10.603631 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2c98a-f8f5-493d-8e09-e400acf6a7ba None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:10.604357 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e2c98a-f8f5-493d-8e09-e400acf6a7ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:10.604669 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e2c98a-f8f5-493d-8e09-e400acf6a7ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:10.606044 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e2c98a-f8f5-493d-8e09-e400acf6a7ba None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:10.615340 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 29/54] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:10.620718 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b457ad8c-3cd9-40e1-b88c-c5a7aed9a496 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:10.682830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:10.719289 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b457ad8c-3cd9-40e1-b88c-c5a7aed9a496 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:10.752868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da2cc7-0a8a-4df0-bd1b-d8c0d0309a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:10.753507 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da2cc7-0a8a-4df0-bd1b-d8c0d0309a3e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:10.753791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da2cc7-0a8a-4df0-bd1b-d8c0d0309a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:10.754068 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da2cc7-0a8a-4df0-bd1b-d8c0d0309a3e None admin] PATH_INFO: `/v3/users/11071bdab076477385007310e6ab9852` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:10.755223 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84da2cc7-0a8a-4df0-bd1b-d8c0d0309a3e None admin] RBAC: Authorizing `identity:get_user(user_id=11071bdab076477385007310e6ab9852)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:10.756721 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84da2cc7-0a8a-4df0-bd1b-d8c0d0309a3e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:10.758316 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 26/55] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:10 2022] GET /identity/v3/users/11071bdab076477385007310e6ab9852 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:10.763837 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2c98a-f8f5-493d-8e09-e400acf6a7ba None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:10.823682 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:10.863151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2c98a-f8f5-493d-8e09-e400acf6a7ba None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:10.898233 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba07f98-394d-4ac2-aa04-babc8f4e0610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:10.898886 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba07f98-394d-4ac2-aa04-babc8f4e0610 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:10.899176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba07f98-394d-4ac2-aa04-babc8f4e0610 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:10.899488 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba07f98-394d-4ac2-aa04-babc8f4e0610 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:10.900630 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba07f98-394d-4ac2-aa04-babc8f4e0610 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:10.902069 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba07f98-394d-4ac2-aa04-babc8f4e0610 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:10.903699 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 30/56] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:29:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:10.909972 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da2cc7-0a8a-4df0-bd1b-d8c0d0309a3e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:11.005705 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da2cc7-0a8a-4df0-bd1b-d8c0d0309a3e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:11.034486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:11.039357 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b66d4-5eed-49d4-b35e-579ad2a9a13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:11.041711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b66d4-5eed-49d4-b35e-579ad2a9a13d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:11.041711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b66d4-5eed-49d4-b35e-579ad2a9a13d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:11.041711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b66d4-5eed-49d4-b35e-579ad2a9a13d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:11.041711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08b66d4-5eed-49d4-b35e-579ad2a9a13d None admin] RBAC: Adding query filter params (role.id=51b2d10dfffb4b75b186f3d8b15e2a78, scope.domain.id=default, user.id=11071bdab076477385007310e6ab9852) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:11.041711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08b66d4-5eed-49d4-b35e-579ad2a9a13d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:11.043073 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08b66d4-5eed-49d4-b35e-579ad2a9a13d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:11.053765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 27/57] 10.222.0.97 () {56 vars in 1322 bytes} [Tue Jul 26 14:29:10 2022] GET /identity/v3/role_assignments?user.id=11071bdab076477385007310e6ab9852&scope.domain.id=default&role.id=51b2d10dfffb4b75b186f3d8b15e2a78 => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:12.422783 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50148b2-7eb7-471d-94cf-1447fe64803a None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:12.423284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50148b2-7eb7-471d-94cf-1447fe64803a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:12.423703 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50148b2-7eb7-471d-94cf-1447fe64803a None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:12.425588 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 31/58] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:12.430675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ee408-f905-4585-8373-29c2d7eaec76 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:12.431092 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ee408-f905-4585-8373-29c2d7eaec76 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:12.431241 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ee408-f905-4585-8373-29c2d7eaec76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:12.449481 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d99ee408-f905-4585-8373-29c2d7eaec76 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:12.450670 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d99ee408-f905-4585-8373-29c2d7eaec76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:12.516874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 28/59] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:12.540937 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50148b2-7eb7-471d-94cf-1447fe64803a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:12.640510 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50148b2-7eb7-471d-94cf-1447fe64803a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:12.666232 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:12.680753 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1faa3-1d82-43cb-8b9c-5737c843ee44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:12.680753 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1faa3-1d82-43cb-8b9c-5737c843ee44 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:12.680753 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1faa3-1d82-43cb-8b9c-5737c843ee44 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:12.680753 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1faa3-1d82-43cb-8b9c-5737c843ee44 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:12.681141 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b1faa3-1d82-43cb-8b9c-5737c843ee44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:12.681141 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b1faa3-1d82-43cb-8b9c-5737c843ee44 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:12.681141 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 32/60] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:29:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:12.684967 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ee408-f905-4585-8373-29c2d7eaec76 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:12.781061 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ee408-f905-4585-8373-29c2d7eaec76 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:12.809236 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:12.818061 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d501ae5-bf1b-4a44-a5ca-1e42aaa29df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:12.818684 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d501ae5-bf1b-4a44-a5ca-1e42aaa29df2 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:12.818974 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d501ae5-bf1b-4a44-a5ca-1e42aaa29df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:12.819259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d501ae5-bf1b-4a44-a5ca-1e42aaa29df2 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:12.819983 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d501ae5-bf1b-4a44-a5ca-1e42aaa29df2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:12.821440 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d501ae5-bf1b-4a44-a5ca-1e42aaa29df2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:12.941480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 29/61] 10.222.0.97 () {60 vars in 1132 bytes} [Tue Jul 26 14:29:12 2022] POST /identity/v3/users => generated 275 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:29:16.067799 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705cd816-f339-4c28-ad3f-90a50d4756f9 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:16.068754 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705cd816-f339-4c28-ad3f-90a50d4756f9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:16.069264 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705cd816-f339-4c28-ad3f-90a50d4756f9 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:16.070863 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 33/62] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:16.076104 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae5b19-09c3-48ec-9af7-9dd3efe56a51 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:16.077526 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae5b19-09c3-48ec-9af7-9dd3efe56a51 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:16.078019 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae5b19-09c3-48ec-9af7-9dd3efe56a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:16.108957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-e7ae5b19-09c3-48ec-9af7-9dd3efe56a51 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:16.110511 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e7ae5b19-09c3-48ec-9af7-9dd3efe56a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:16.182604 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfdac8e-358d-4d06-9eac-6532908f057a None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:16.183259 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfdac8e-358d-4d06-9eac-6532908f057a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:16.183782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfdac8e-358d-4d06-9eac-6532908f057a None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:16.185373 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 34/63] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:16.190886 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90462de1-cdbc-4f00-9705-462c0d54264f None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:16.191187 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90462de1-cdbc-4f00-9705-462c0d54264f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:16.191497 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90462de1-cdbc-4f00-9705-462c0d54264f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:16.222300 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 30/64] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:16.222497 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-90462de1-cdbc-4f00-9705-462c0d54264f None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:16.223816 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-90462de1-cdbc-4f00-9705-462c0d54264f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:16.245577 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae5b19-09c3-48ec-9af7-9dd3efe56a51 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:16.354422 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 35/65] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:29:16.355521 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913d699-cb3d-47ea-8fa5-2d4072083713 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:16.355819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913d699-cb3d-47ea-8fa5-2d4072083713 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:16.356140 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913d699-cb3d-47ea-8fa5-2d4072083713 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:16.357411 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 36/66] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:16.368707 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0cc0c-d0f5-4263-ab41-7413c68c64ac None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:16.369264 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0cc0c-d0f5-4263-ab41-7413c68c64ac None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:16.369741 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0cc0c-d0f5-4263-ab41-7413c68c64ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:16.375370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae5b19-09c3-48ec-9af7-9dd3efe56a51 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:16.389828 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:16.394114 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-09f0cc0c-d0f5-4263-ab41-7413c68c64ac None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:16.395315 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-09f0cc0c-d0f5-4263-ab41-7413c68c64ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:16.413879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b431c61c-c3dc-41af-85f3-3d2228b04f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:16.414638 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b431c61c-c3dc-41af-85f3-3d2228b04f26 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:16.414928 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b431c61c-c3dc-41af-85f3-3d2228b04f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:16.415209 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b431c61c-c3dc-41af-85f3-3d2228b04f26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:16.426649 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b431c61c-c3dc-41af-85f3-3d2228b04f26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:16.432711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-b431c61c-c3dc-41af-85f3-3d2228b04f26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:29:16.432711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:16.432711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:16.432711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:16.432711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:16.432711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:16.432711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:16.432711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:16.432711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:16.432711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:16.432711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:16.432711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:16.432711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:16.432711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:16.432711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:16.432711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:16.433644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:16.433644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:16.433644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:16.433644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:16.433644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:16.433644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:16.433644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:16.433644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:16.433644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:16.433644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:16.433644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:16.433644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:16.433644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:16.433644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:16.433644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:16.433644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:16.434422 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:16.434422 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:16.434422 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:16.434422 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:16.434422 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:16.434422 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:16.434422 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:16.434422 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:16.434422 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:16.434422 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:16.434422 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:29:16.434422 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:29:16.439690 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 31/67] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:29:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:16.441308 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b431c61c-c3dc-41af-85f3-3d2228b04f26 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:16.477534 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 37/68] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:16.478607 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0cc0c-d0f5-4263-ab41-7413c68c64ac None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:16.542788 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:16.550685 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b431c61c-c3dc-41af-85f3-3d2228b04f26 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:16.585894 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a5009-c285-4561-9c30-4704a1280800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:16.586560 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a5009-c285-4561-9c30-4704a1280800 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:16.586802 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0cc0c-d0f5-4263-ab41-7413c68c64ac None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:16.586919 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a5009-c285-4561-9c30-4704a1280800 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:16.587106 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a5009-c285-4561-9c30-4704a1280800 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:16.599295 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a5009-c285-4561-9c30-4704a1280800 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:16.604600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-ee5a5009-c285-4561-9c30-4704a1280800 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:16.604600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:16.604600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:16.604600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:16.604600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:16.604600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:16.604600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:16.604600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:16.604600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:16.604600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:16.604600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:16.604600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:16.604600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:16.604600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:16.604600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:16.604600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:16.605462 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:16.605462 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:16.605462 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:16.605462 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:16.605462 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:16.605462 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:16.605462 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:16.605462 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:16.605462 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:16.605462 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:16.605462 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:16.605462 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:16.605462 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:16.605462 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:16.605462 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:16.605462 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:16.606193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:16.606193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:16.606193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:16.606193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:16.606193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:16.606193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:16.606193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:16.606193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:16.606193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:16.606193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:16.606193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:16.606193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:29:16.606193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 32/69] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:16.607453 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bf57c-be49-4791-8030-5bd1906ceb6e None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:16.607453 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bf57c-be49-4791-8030-5bd1906ceb6e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:16.607615 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bf57c-be49-4791-8030-5bd1906ceb6e None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:16.609311 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 33/70] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:16.610005 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bf57c-be49-4791-8030-5bd1906ceb6e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:16.634647 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf322ce-e77b-478e-a9e1-536abb58e1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:16.635275 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf322ce-e77b-478e-a9e1-536abb58e1af None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:16.635606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf322ce-e77b-478e-a9e1-536abb58e1af None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:16.635886 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf322ce-e77b-478e-a9e1-536abb58e1af None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:16.636612 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf322ce-e77b-478e-a9e1-536abb58e1af None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:16.636919 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf322ce-e77b-478e-a9e1-536abb58e1af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:16.638300 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf322ce-e77b-478e-a9e1-536abb58e1af None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:16.644759 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:16.647781 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 38/71] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:16.652497 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf322ce-e77b-478e-a9e1-536abb58e1af None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:16.726081 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bf57c-be49-4791-8030-5bd1906ceb6e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:16.753165 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf322ce-e77b-478e-a9e1-536abb58e1af None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:16.761282 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca56011-f76a-4312-a948-56e6b5ceb961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:16.761934 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca56011-f76a-4312-a948-56e6b5ceb961 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:16.762228 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca56011-f76a-4312-a948-56e6b5ceb961 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:16.762513 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca56011-f76a-4312-a948-56e6b5ceb961 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:16.769990 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:16.772763 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca56011-f76a-4312-a948-56e6b5ceb961 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:16.777717 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-0ca56011-f76a-4312-a948-56e6b5ceb961 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:29:16.777717 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:16.777717 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:16.777717 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:16.777717 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:16.777717 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:16.777717 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:16.777717 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:16.777717 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:16.777717 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:16.777717 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:16.777717 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:16.777717 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:16.777717 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:16.777717 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:16.777717 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:16.782431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:16.782431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:16.782431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:16.782431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:16.782431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:16.782431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:16.782431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:16.782431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:16.782431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:16.782431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:16.782431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:16.782431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:16.782431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:16.782431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:16.782431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:16.782431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:16.783370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:16.783370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:16.783370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:16.783370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:16.783370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:16.783370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:16.783370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:16.783370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:16.783370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:16.783370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:16.783370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:29:16.783370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:29:16.783370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 34/72] 10.222.0.97 () {56 vars in 1114 bytes} [Tue Jul 26 14:29:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:16.783370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f6a40-77ea-4a55-a955-8ac658a9c419 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:16.784205 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f6a40-77ea-4a55-a955-8ac658a9c419 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:16.784205 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f6a40-77ea-4a55-a955-8ac658a9c419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:16.790805 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd7482-5e9f-45f1-a225-6cbca90c798d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:16.791465 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd7482-5e9f-45f1-a225-6cbca90c798d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:16.791757 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd7482-5e9f-45f1-a225-6cbca90c798d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:16.792055 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd7482-5e9f-45f1-a225-6cbca90c798d None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:16.792807 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd7482-5e9f-45f1-a225-6cbca90c798d None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:16.793117 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd7482-5e9f-45f1-a225-6cbca90c798d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:16.794483 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd7482-5e9f-45f1-a225-6cbca90c798d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:16.799086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-497f6a40-77ea-4a55-a955-8ac658a9c419 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:16.800333 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-497f6a40-77ea-4a55-a955-8ac658a9c419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:16.803686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 39/73] 10.222.0.97 () {56 vars in 1101 bytes} [Tue Jul 26 14:29:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:16.808637 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd7482-5e9f-45f1-a225-6cbca90c798d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:16.871864 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 35/74] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:16.873388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817abad6-5449-4391-bbbe-6e45bdf42e33 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:16.873680 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817abad6-5449-4391-bbbe-6e45bdf42e33 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:16.873961 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817abad6-5449-4391-bbbe-6e45bdf42e33 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:16.875216 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 36/75] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:16.879234 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817abad6-5449-4391-bbbe-6e45bdf42e33 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:16.905526 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:16.917532 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd7482-5e9f-45f1-a225-6cbca90c798d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:16.951865 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab116a21-2bd2-4a06-86b1-21219644afdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:16.952519 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab116a21-2bd2-4a06-86b1-21219644afdf None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:16.952822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab116a21-2bd2-4a06-86b1-21219644afdf None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:16.953105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab116a21-2bd2-4a06-86b1-21219644afdf None admin] PATH_INFO: `/v3/users/11071bdab076477385007310e6ab9852` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:16.954295 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab116a21-2bd2-4a06-86b1-21219644afdf None admin] RBAC: Authorizing `identity:get_user(user_id=11071bdab076477385007310e6ab9852)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:16.955770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab116a21-2bd2-4a06-86b1-21219644afdf None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:16.957381 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 40/76] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:16 2022] GET /identity/v3/users/11071bdab076477385007310e6ab9852 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:16.958137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab116a21-2bd2-4a06-86b1-21219644afdf None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:16.981330 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817abad6-5449-4391-bbbe-6e45bdf42e33 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:17.008240 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:17.016864 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df8ce5-1029-488a-a726-75d14f01a058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:17.022448 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df8ce5-1029-488a-a726-75d14f01a058 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:17.022448 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df8ce5-1029-488a-a726-75d14f01a058 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:17.022448 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df8ce5-1029-488a-a726-75d14f01a058 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:17.022448 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df8ce5-1029-488a-a726-75d14f01a058 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:17.022448 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df8ce5-1029-488a-a726-75d14f01a058 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:17.022448 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df8ce5-1029-488a-a726-75d14f01a058 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:17.028744 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 37/77] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:29:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:17.031467 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea93e8-1bda-4bf9-be28-801bf97ac770 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:17.031467 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea93e8-1bda-4bf9-be28-801bf97ac770 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:17.031467 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea93e8-1bda-4bf9-be28-801bf97ac770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:17.048932 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0eea93e8-1bda-4bf9-be28-801bf97ac770 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:17.050154 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0eea93e8-1bda-4bf9-be28-801bf97ac770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:17.063903 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab116a21-2bd2-4a06-86b1-21219644afdf None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:17.099062 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e237bbe-b473-49ef-92bd-24c09aad2e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:17.099725 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e237bbe-b473-49ef-92bd-24c09aad2e64 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:17.100020 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e237bbe-b473-49ef-92bd-24c09aad2e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:17.100303 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e237bbe-b473-49ef-92bd-24c09aad2e64 None admin] PATH_INFO: `/v3/users/45da4bf9f6e64bc9bfa6f4e5beafb859` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:17.115626 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e237bbe-b473-49ef-92bd-24c09aad2e64 None admin] RBAC: Authorizing `identity:get_user(user_id=45da4bf9f6e64bc9bfa6f4e5beafb859)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:17.118114 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e237bbe-b473-49ef-92bd-24c09aad2e64 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:17.118540 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 38/78] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:17.119389 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea93e8-1bda-4bf9-be28-801bf97ac770 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:17.125624 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 41/79] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:16 2022] GET /identity/v3/users/45da4bf9f6e64bc9bfa6f4e5beafb859 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:17.125624 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e237bbe-b473-49ef-92bd-24c09aad2e64 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:17.140316 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:17.224404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea93e8-1bda-4bf9-be28-801bf97ac770 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:17.224628 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e237bbe-b473-49ef-92bd-24c09aad2e64 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:17.231924 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:17.257631 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c2fd5-1a94-4443-8c24-b1982e45a521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:17.258143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424beeec-214f-4375-adf7-e3d0b98054af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:17.258292 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c2fd5-1a94-4443-8c24-b1982e45a521 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:17.258547 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c2fd5-1a94-4443-8c24-b1982e45a521 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:17.258834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c2fd5-1a94-4443-8c24-b1982e45a521 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:17.259371 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424beeec-214f-4375-adf7-e3d0b98054af None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:17.259694 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424beeec-214f-4375-adf7-e3d0b98054af None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:17.259979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424beeec-214f-4375-adf7-e3d0b98054af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:17.260079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c2fd5-1a94-4443-8c24-b1982e45a521 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:17.261445 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c2fd5-1a94-4443-8c24-b1982e45a521 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:17.263040 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 39/80] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:29:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:17.263824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c2fd5-1a94-4443-8c24-b1982e45a521 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:17.329013 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424beeec-214f-4375-adf7-e3d0b98054af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:17.330483 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424beeec-214f-4375-adf7-e3d0b98054af None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:17.344138 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:17.362812 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c2fd5-1a94-4443-8c24-b1982e45a521 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:17.397013 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52aea41-13ca-4f33-b42c-c21df6659477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:17.397584 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52aea41-13ca-4f33-b42c-c21df6659477 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:17.397875 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52aea41-13ca-4f33-b42c-c21df6659477 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:17.398159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52aea41-13ca-4f33-b42c-c21df6659477 None admin] PATH_INFO: `/v3/users/45da4bf9f6e64bc9bfa6f4e5beafb859` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:17.399392 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52aea41-13ca-4f33-b42c-c21df6659477 None admin] RBAC: Authorizing `identity:get_user(user_id=45da4bf9f6e64bc9bfa6f4e5beafb859)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:17.400875 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52aea41-13ca-4f33-b42c-c21df6659477 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:17.402495 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 40/81] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:17 2022] GET /identity/v3/users/45da4bf9f6e64bc9bfa6f4e5beafb859 => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:17.402495 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52aea41-13ca-4f33-b42c-c21df6659477 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:17.429638 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 42/82] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:29:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:29:17.430547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424beeec-214f-4375-adf7-e3d0b98054af None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:17.439560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:17.505980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52aea41-13ca-4f33-b42c-c21df6659477 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:17.528501 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424beeec-214f-4375-adf7-e3d0b98054af None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:17.541490 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b834b0e1-a677-4170-b013-6a8d3185c974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:17.542545 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834b0e1-a677-4170-b013-6a8d3185c974 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:17.543036 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834b0e1-a677-4170-b013-6a8d3185c974 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:17.543574 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834b0e1-a677-4170-b013-6a8d3185c974 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:17.562739 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308865f-49ba-4735-998c-83a397c0e243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:17.563442 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308865f-49ba-4735-998c-83a397c0e243 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:17.563709 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308865f-49ba-4735-998c-83a397c0e243 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:17.563987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308865f-49ba-4735-998c-83a397c0e243 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:17.564944 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:17.574197 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7308865f-49ba-4735-998c-83a397c0e243 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:17.579537 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-7308865f-49ba-4735-998c-83a397c0e243 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:17.579537 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:17.579537 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:17.579537 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:17.579537 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:17.579537 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:17.579537 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:17.579537 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:17.579537 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:17.579537 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:17.579537 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:17.579537 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:17.579537 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:17.579537 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:17.579537 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:17.579537 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:17.580498 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:17.580498 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:17.580498 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:17.580498 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:17.580498 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:17.580498 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:17.580498 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:17.580498 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:17.580498 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:17.580498 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:17.580498 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:17.580498 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:17.580498 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:17.580498 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:17.580498 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:17.580498 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:17.581424 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:17.581424 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:17.581424 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:17.581424 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:17.581424 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:17.581424 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:17.581424 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:17.581424 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:17.581424 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:17.581424 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:17.581424 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:17.581424 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:29:17.581424 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 43/83] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:17.582047 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308865f-49ba-4735-998c-83a397c0e243 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:17.615983 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b834b0e1-a677-4170-b013-6a8d3185c974 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:17.617475 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b834b0e1-a677-4170-b013-6a8d3185c974 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:17.646826 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:17.680502 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308865f-49ba-4735-998c-83a397c0e243 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:17.714896 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47311710-e23a-4e9b-9a11-300994b328af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:17.715546 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47311710-e23a-4e9b-9a11-300994b328af None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:17.715838 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47311710-e23a-4e9b-9a11-300994b328af None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:17.716119 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47311710-e23a-4e9b-9a11-300994b328af None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:17.721453 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 41/84] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:29:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:29:17.721581 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47311710-e23a-4e9b-9a11-300994b328af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:17.721673 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b834b0e1-a677-4170-b013-6a8d3185c974 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:17.721751 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47311710-e23a-4e9b-9a11-300994b328af None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:17.765574 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:17.792921 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 44/85] 10.222.0.97 () {60 vars in 1132 bytes} [Tue Jul 26 14:29:17 2022] POST /identity/v3/users => generated 289 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:29:17.793821 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47311710-e23a-4e9b-9a11-300994b328af None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:17.816056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b834b0e1-a677-4170-b013-6a8d3185c974 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:17.864605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141313ee-e252-4b41-889d-1e39c3eeb8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:17.865274 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141313ee-e252-4b41-889d-1e39c3eeb8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:17.865567 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141313ee-e252-4b41-889d-1e39c3eeb8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:17.865851 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141313ee-e252-4b41-889d-1e39c3eeb8c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:17.928002 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47311710-e23a-4e9b-9a11-300994b328af None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:17.946613 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141313ee-e252-4b41-889d-1e39c3eeb8c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:17.946613 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141313ee-e252-4b41-889d-1e39c3eeb8c6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:17.957217 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:17.972758 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66cf464-b200-4c59-a0d4-4bcc3cf5510c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:17.973455 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cf464-b200-4c59-a0d4-4bcc3cf5510c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:17.974050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cf464-b200-4c59-a0d4-4bcc3cf5510c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:17.974050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cf464-b200-4c59-a0d4-4bcc3cf5510c None admin] PATH_INFO: `/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:17.975469 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66cf464-b200-4c59-a0d4-4bcc3cf5510c None admin] RBAC: Authorizing `identity:get_project(project_id=076f7405c1874c1ea0d0f44c11ae4d38)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:17.976847 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66cf464-b200-4c59-a0d4-4bcc3cf5510c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:17.978706 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 45/86] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:17 2022] GET /identity/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:17.979530 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66cf464-b200-4c59-a0d4-4bcc3cf5510c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:18.002305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:18.066530 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 42/87] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:29:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:29:18.072454 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141313ee-e252-4b41-889d-1e39c3eeb8c6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:18.107002 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66cf464-b200-4c59-a0d4-4bcc3cf5510c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:18.146304 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e81e0-c400-40da-ae30-fe820860c7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:18.146941 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e81e0-c400-40da-ae30-fe820860c7ab None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:18.147252 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e81e0-c400-40da-ae30-fe820860c7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:18.147568 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e81e0-c400-40da-ae30-fe820860c7ab None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:18.148302 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e81e0-c400-40da-ae30-fe820860c7ab None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:18.148662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e81e0-c400-40da-ae30-fe820860c7ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:18.150163 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e81e0-c400-40da-ae30-fe820860c7ab None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:18.164379 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 46/88] 10.222.0.97 () {56 vars in 1101 bytes} [Tue Jul 26 14:29:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:18.165439 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e81e0-c400-40da-ae30-fe820860c7ab None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:18.264412 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141313ee-e252-4b41-889d-1e39c3eeb8c6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:18.293975 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:18.308317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea5ea2-b0b0-412c-bb88-37629acc0d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:18.310316 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea5ea2-b0b0-412c-bb88-37629acc0d21 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:18.310316 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea5ea2-b0b0-412c-bb88-37629acc0d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:18.310316 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea5ea2-b0b0-412c-bb88-37629acc0d21 None admin] PATH_INFO: `/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:18.310887 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ea5ea2-b0b0-412c-bb88-37629acc0d21 None admin] RBAC: Authorizing `identity:get_project(project_id=076f7405c1874c1ea0d0f44c11ae4d38)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:18.312863 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ea5ea2-b0b0-412c-bb88-37629acc0d21 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:18.314212 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 43/89] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:18 2022] GET /identity/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38 => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:18.314916 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea5ea2-b0b0-412c-bb88-37629acc0d21 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:18.335546 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:18.349985 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e81e0-c400-40da-ae30-fe820860c7ab None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:18.387698 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f21b3-8d25-4557-9b18-3ee9e3c11e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:18.388680 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f21b3-8d25-4557-9b18-3ee9e3c11e76 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:18.389712 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f21b3-8d25-4557-9b18-3ee9e3c11e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:18.390198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f21b3-8d25-4557-9b18-3ee9e3c11e76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:18.391193 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f21b3-8d25-4557-9b18-3ee9e3c11e76 None admin] RBAC: Adding query filter params (role.id=51b2d10dfffb4b75b186f3d8b15e2a78, scope.project.id=076f7405c1874c1ea0d0f44c11ae4d38, user.id=11071bdab076477385007310e6ab9852) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:18.391733 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f21b3-8d25-4557-9b18-3ee9e3c11e76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:18.393213 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f21b3-8d25-4557-9b18-3ee9e3c11e76 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:18.399391 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 47/90] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:29:18 2022] GET /identity/v3/role_assignments?user.id=11071bdab076477385007310e6ab9852&scope.project.id=076f7405c1874c1ea0d0f44c11ae4d38&role.id=51b2d10dfffb4b75b186f3d8b15e2a78 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:18.400257 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f21b3-8d25-4557-9b18-3ee9e3c11e76 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:18.432283 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea5ea2-b0b0-412c-bb88-37629acc0d21 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:18.494840 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc3642-9bf2-4d9a-bcd6-7de208d36f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:18.495537 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc3642-9bf2-4d9a-bcd6-7de208d36f6a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:18.495831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc3642-9bf2-4d9a-bcd6-7de208d36f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:18.496116 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc3642-9bf2-4d9a-bcd6-7de208d36f6a None admin] PATH_INFO: `/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:18.497461 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc3642-9bf2-4d9a-bcd6-7de208d36f6a None admin] RBAC: Authorizing `identity:get_project(project_id=076f7405c1874c1ea0d0f44c11ae4d38)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:18.499502 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc3642-9bf2-4d9a-bcd6-7de208d36f6a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:18.503864 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 44/91] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:18 2022] GET /identity/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:18.504594 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc3642-9bf2-4d9a-bcd6-7de208d36f6a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:18.602095 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f21b3-8d25-4557-9b18-3ee9e3c11e76 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:18.622919 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:18.644559 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:18.645711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1633101b-7b9c-44cc-9e13-53fb43391b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:18.646582 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633101b-7b9c-44cc-9e13-53fb43391b1c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:18.647059 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633101b-7b9c-44cc-9e13-53fb43391b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:18.648095 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633101b-7b9c-44cc-9e13-53fb43391b1c None admin] PATH_INFO: `/v3/users/45da4bf9f6e64bc9bfa6f4e5beafb859` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:18.649652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1633101b-7b9c-44cc-9e13-53fb43391b1c None admin] RBAC: Authorizing `identity:get_user(user_id=45da4bf9f6e64bc9bfa6f4e5beafb859)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:18.651943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1633101b-7b9c-44cc-9e13-53fb43391b1c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:18.654239 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 48/92] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:18 2022] GET /identity/v3/users/45da4bf9f6e64bc9bfa6f4e5beafb859 => generated 275 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:18.659441 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1633101b-7b9c-44cc-9e13-53fb43391b1c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:18.667349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc3642-9bf2-4d9a-bcd6-7de208d36f6a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:18.742841 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816aa1bf-870d-4022-a675-4fcc1b6bc453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:18.744024 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816aa1bf-870d-4022-a675-4fcc1b6bc453 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:18.745311 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816aa1bf-870d-4022-a675-4fcc1b6bc453 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:18.745788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816aa1bf-870d-4022-a675-4fcc1b6bc453 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:18.751800 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816aa1bf-870d-4022-a675-4fcc1b6bc453 None admin] RBAC: Adding query filter params (role.id=e033961a9363456a8510c34414dd839e, scope.project.id=076f7405c1874c1ea0d0f44c11ae4d38, user.id=45da4bf9f6e64bc9bfa6f4e5beafb859) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:18.752340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816aa1bf-870d-4022-a675-4fcc1b6bc453 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:18.754039 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816aa1bf-870d-4022-a675-4fcc1b6bc453 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:18.762271 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 45/93] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:29:18 2022] GET /identity/v3/role_assignments?user.id=45da4bf9f6e64bc9bfa6f4e5beafb859&scope.project.id=076f7405c1874c1ea0d0f44c11ae4d38&role.id=e033961a9363456a8510c34414dd839e => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:18.765319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816aa1bf-870d-4022-a675-4fcc1b6bc453 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:18.793591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1633101b-7b9c-44cc-9e13-53fb43391b1c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:18.835727 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be49d2-2f97-47b9-8fa1-e74e21f68f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:18.837341 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be49d2-2f97-47b9-8fa1-e74e21f68f80 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:18.838249 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be49d2-2f97-47b9-8fa1-e74e21f68f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:18.838916 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be49d2-2f97-47b9-8fa1-e74e21f68f80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:18.840259 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04be49d2-2f97-47b9-8fa1-e74e21f68f80 None admin] RBAC: Adding query filter params (role.id=0602cf371b0f41b390e8a4499ae6bb3c, scope.project.id=076f7405c1874c1ea0d0f44c11ae4d38, user.id=45da4bf9f6e64bc9bfa6f4e5beafb859) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:18.841014 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04be49d2-2f97-47b9-8fa1-e74e21f68f80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:18.845920 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04be49d2-2f97-47b9-8fa1-e74e21f68f80 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:18.858693 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 49/94] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:29:18 2022] GET /identity/v3/role_assignments?user.id=45da4bf9f6e64bc9bfa6f4e5beafb859&scope.project.id=076f7405c1874c1ea0d0f44c11ae4d38&role.id=0602cf371b0f41b390e8a4499ae6bb3c => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:18.970589 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816aa1bf-870d-4022-a675-4fcc1b6bc453 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:19.019536 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e00539-8aa5-43cb-8d06-fa9f5696ec77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:19.020178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e00539-8aa5-43cb-8d06-fa9f5696ec77 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:19.020469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e00539-8aa5-43cb-8d06-fa9f5696ec77 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:19.020752 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e00539-8aa5-43cb-8d06-fa9f5696ec77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:19.059265 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:19.116043 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e00539-8aa5-43cb-8d06-fa9f5696ec77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:19.117438 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e00539-8aa5-43cb-8d06-fa9f5696ec77 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:19.239013 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 46/95] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:29:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:29:19.245422 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be49d2-2f97-47b9-8fa1-e74e21f68f80 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:19.329800 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:19.397423 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be49d2-2f97-47b9-8fa1-e74e21f68f80 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:19.445038 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c4262f-480e-4be1-b5e3-b3a2580d050c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:19.445708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4262f-480e-4be1-b5e3-b3a2580d050c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:19.445994 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4262f-480e-4be1-b5e3-b3a2580d050c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:19.446276 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4262f-480e-4be1-b5e3-b3a2580d050c None admin] PATH_INFO: `/v3/projects/8d0806a2c77d415391f06dc0ed0f21d2` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:19.447483 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c4262f-480e-4be1-b5e3-b3a2580d050c None admin] RBAC: Authorizing `identity:get_project(project_id=8d0806a2c77d415391f06dc0ed0f21d2)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:19.449018 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c4262f-480e-4be1-b5e3-b3a2580d050c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:19.450731 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 50/96] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:19 2022] GET /identity/v3/projects/8d0806a2c77d415391f06dc0ed0f21d2 => generated 312 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:19.456956 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e00539-8aa5-43cb-8d06-fa9f5696ec77 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:19.517792 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:19.579827 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e00539-8aa5-43cb-8d06-fa9f5696ec77 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:19.624108 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0866dd-d509-4324-adb5-d91c1e62d1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:19.624740 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0866dd-d509-4324-adb5-d91c1e62d1ad None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:19.625031 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0866dd-d509-4324-adb5-d91c1e62d1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:19.625309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0866dd-d509-4324-adb5-d91c1e62d1ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:19.626112 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0866dd-d509-4324-adb5-d91c1e62d1ad None admin] RBAC: Adding query filter params (role.id=e033961a9363456a8510c34414dd839e, scope.project.id=8d0806a2c77d415391f06dc0ed0f21d2, user.id=45da4bf9f6e64bc9bfa6f4e5beafb859) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:19.626428 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0866dd-d509-4324-adb5-d91c1e62d1ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:19.627901 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0866dd-d509-4324-adb5-d91c1e62d1ad None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:19.633383 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 47/97] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:29:19 2022] GET /identity/v3/role_assignments?user.id=45da4bf9f6e64bc9bfa6f4e5beafb859&scope.project.id=8d0806a2c77d415391f06dc0ed0f21d2&role.id=e033961a9363456a8510c34414dd839e => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:20.589841 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218cd7a0-0aca-4a00-aada-abfed10cbcc3 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:20.590397 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218cd7a0-0aca-4a00-aada-abfed10cbcc3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:20.590397 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218cd7a0-0aca-4a00-aada-abfed10cbcc3 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:20.592045 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 51/98] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:20.597100 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c2789-c41d-4209-8779-fe48c2b5ee2d None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:20.597506 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c2789-c41d-4209-8779-fe48c2b5ee2d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:20.597670 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c2789-c41d-4209-8779-fe48c2b5ee2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:20.616286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-412c2789-c41d-4209-8779-fe48c2b5ee2d None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:20.617487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-412c2789-c41d-4209-8779-fe48c2b5ee2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:20.690557 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13427abf-2a30-495c-bac7-0303d7c0df19 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:20.690894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13427abf-2a30-495c-bac7-0303d7c0df19 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:20.691123 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13427abf-2a30-495c-bac7-0303d7c0df19 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:20.692653 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 52/99] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:29:20.700316 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c2a3c-b622-400d-b056-a4ceb8832649 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:20.700616 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c2a3c-b622-400d-b056-a4ceb8832649 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:20.700877 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c2a3c-b622-400d-b056-a4ceb8832649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:20.716372 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 48/100] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:20.724611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a32c2a3c-b622-400d-b056-a4ceb8832649 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:20.725808 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a32c2a3c-b622-400d-b056-a4ceb8832649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:20.797786 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c2789-c41d-4209-8779-fe48c2b5ee2d None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:20.828467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 53/101] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:29:20.851676 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:20.863126 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32c2a3c-b622-400d-b056-a4ceb8832649 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:20.963934 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:20.982517 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c2789-c41d-4209-8779-fe48c2b5ee2d None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:21.069259 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32c2a3c-b622-400d-b056-a4ceb8832649 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:21.076650 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be3907f-a006-49dd-99dc-f8323cbf7dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:21.077338 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3907f-a006-49dd-99dc-f8323cbf7dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:21.077628 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3907f-a006-49dd-99dc-f8323cbf7dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:21.077910 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3907f-a006-49dd-99dc-f8323cbf7dd1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:21.106771 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f2d87c-aa46-450f-9930-b546eee5841a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:21.107886 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2d87c-aa46-450f-9930-b546eee5841a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:21.108318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2d87c-aa46-450f-9930-b546eee5841a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:21.108604 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2d87c-aa46-450f-9930-b546eee5841a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:21.108741 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be3907f-a006-49dd-99dc-f8323cbf7dd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:21.109774 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f2d87c-aa46-450f-9930-b546eee5841a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:21.111240 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f2d87c-aa46-450f-9930-b546eee5841a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:21.112883 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 54/102] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:29:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:21.113867 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-1be3907f-a006-49dd-99dc-f8323cbf7dd1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:29:21.113867 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:21.113867 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:21.113867 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:21.113867 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:21.113867 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:21.113867 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:21.113867 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:21.113867 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:21.113867 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:21.113867 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:21.113867 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:21.113867 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:21.113867 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:21.113867 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:21.113867 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:21.114679 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:21.114679 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:21.114679 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:21.114679 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:21.114679 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:21.114679 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:21.114679 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:21.114679 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:21.114679 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:21.114679 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:21.114679 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:21.114679 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:21.114679 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:21.114679 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:21.114679 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:21.114679 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:21.115456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:21.115456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:21.115456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:21.115456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:21.115456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:21.115456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:21.115456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:21.115456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:21.115456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:21.115456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:21.115456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:29:21.115456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:29:21.116031 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03933e8c-5913-48f2-a2dc-3611f38f6bce None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:21.116031 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03933e8c-5913-48f2-a2dc-3611f38f6bce None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:21.116031 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03933e8c-5913-48f2-a2dc-3611f38f6bce None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:21.116193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 49/103] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:29:21.118712 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 55/104] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:21.124379 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03933e8c-5913-48f2-a2dc-3611f38f6bce None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:21.130869 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be3907f-a006-49dd-99dc-f8323cbf7dd1 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:21.252992 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03933e8c-5913-48f2-a2dc-3611f38f6bce None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:21.271091 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:21.302132 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da142274-7194-4847-a334-89f5c7a60802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:21.302773 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da142274-7194-4847-a334-89f5c7a60802 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:21.303059 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da142274-7194-4847-a334-89f5c7a60802 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:21.303344 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da142274-7194-4847-a334-89f5c7a60802 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:21.304111 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da142274-7194-4847-a334-89f5c7a60802 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:21.304419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da142274-7194-4847-a334-89f5c7a60802 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:21.305802 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da142274-7194-4847-a334-89f5c7a60802 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:21.328768 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 56/105] 10.222.0.97 () {56 vars in 1101 bytes} [Tue Jul 26 14:29:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:21.331162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545a68e-b192-4f81-8cb3-27013b0057cd None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:21.334077 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545a68e-b192-4f81-8cb3-27013b0057cd None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:21.334377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545a68e-b192-4f81-8cb3-27013b0057cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:21.345745 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be3907f-a006-49dd-99dc-f8323cbf7dd1 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:21.354255 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:21.379333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6545a68e-b192-4f81-8cb3-27013b0057cd None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:21.380866 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6545a68e-b192-4f81-8cb3-27013b0057cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:21.393162 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc40e12-53f3-4630-9ada-91e3f6672521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:21.394054 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc40e12-53f3-4630-9ada-91e3f6672521 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:21.394344 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc40e12-53f3-4630-9ada-91e3f6672521 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:21.394629 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc40e12-53f3-4630-9ada-91e3f6672521 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:21.399437 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc40e12-53f3-4630-9ada-91e3f6672521 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:21.399437 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc40e12-53f3-4630-9ada-91e3f6672521 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:21.480690 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 50/106] 10.222.0.97 () {60 vars in 1141 bytes} [Tue Jul 26 14:29:21 2022] POST /identity/v3/projects => generated 302 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:29:21.481606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc40e12-53f3-4630-9ada-91e3f6672521 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:21.504068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 57/107] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:21.562268 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545a68e-b192-4f81-8cb3-27013b0057cd None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:21.664151 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc40e12-53f3-4630-9ada-91e3f6672521 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:21.718338 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29151a6b-2439-4acd-96d5-a125816f9b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:21.719014 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29151a6b-2439-4acd-96d5-a125816f9b09 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:21.719301 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29151a6b-2439-4acd-96d5-a125816f9b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:21.719625 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29151a6b-2439-4acd-96d5-a125816f9b09 None admin] PATH_INFO: `/v3/users/36ceba794e7f467d989738a87a04100c` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:21.736328 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29151a6b-2439-4acd-96d5-a125816f9b09 None admin] RBAC: Authorizing `identity:get_user(user_id=36ceba794e7f467d989738a87a04100c)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:21.737858 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29151a6b-2439-4acd-96d5-a125816f9b09 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:21.740049 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 51/108] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:21 2022] GET /identity/v3/users/36ceba794e7f467d989738a87a04100c => generated 289 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:21.749310 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29151a6b-2439-4acd-96d5-a125816f9b09 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:21.766929 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:21.832037 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545a68e-b192-4f81-8cb3-27013b0057cd None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:21.841553 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:21.873471 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29151a6b-2439-4acd-96d5-a125816f9b09 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:21.875269 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4bc7b5-8afa-42b2-a5dc-24e58fddf800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:21.876034 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4bc7b5-8afa-42b2-a5dc-24e58fddf800 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:21.876400 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4bc7b5-8afa-42b2-a5dc-24e58fddf800 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:21.877608 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4bc7b5-8afa-42b2-a5dc-24e58fddf800 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:21.896038 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4bc7b5-8afa-42b2-a5dc-24e58fddf800 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:21.901780 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-af4bc7b5-8afa-42b2-a5dc-24e58fddf800 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:29:21.901780 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:21.901780 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:21.901780 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:21.901780 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:21.901780 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:21.901780 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:21.901780 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:21.901780 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:21.901780 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:21.901780 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:21.901780 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:21.901780 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:21.901780 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:21.901780 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:21.901780 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:21.902601 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:21.902601 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:21.902601 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:21.902601 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:21.902601 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:21.902601 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:21.902601 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:21.902601 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:21.902601 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:21.902601 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:21.902601 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:21.902601 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:21.902601 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:21.902601 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:21.902601 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:21.902601 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:21.903331 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:21.903331 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:21.903331 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:21.903331 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:21.903331 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:21.903331 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:21.903331 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:21.903331 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:21.903331 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:21.903331 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:21.903331 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:29:21.903331 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:29:21.905945 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 58/109] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:29:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:29:21.907332 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5045c3-6361-4b75-81ac-abfdf4095d81 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:21.907859 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5045c3-6361-4b75-81ac-abfdf4095d81 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:21.908388 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5045c3-6361-4b75-81ac-abfdf4095d81 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:21.909954 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 59/110] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:21.912933 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5045c3-6361-4b75-81ac-abfdf4095d81 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:21.926415 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddebea-e415-4015-bd6c-8f3f52852875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:21.926415 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddebea-e415-4015-bd6c-8f3f52852875 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:21.926415 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddebea-e415-4015-bd6c-8f3f52852875 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:21.926704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddebea-e415-4015-bd6c-8f3f52852875 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:22.066291 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ddebea-e415-4015-bd6c-8f3f52852875 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:22.068027 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ddebea-e415-4015-bd6c-8f3f52852875 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:22.075972 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5045c3-6361-4b75-81ac-abfdf4095d81 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:22.087295 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:22.122759 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c08d22-677d-4783-a998-b1c0050f37e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:22.123490 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c08d22-677d-4783-a998-b1c0050f37e2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:22.123784 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c08d22-677d-4783-a998-b1c0050f37e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:22.124211 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c08d22-677d-4783-a998-b1c0050f37e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:22.125347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c08d22-677d-4783-a998-b1c0050f37e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:22.126127 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c08d22-677d-4783-a998-b1c0050f37e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:22.128006 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c08d22-677d-4783-a998-b1c0050f37e2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:22.140190 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 60/111] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:22.141435 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a42c18-50cc-44f6-a121-90ec6367b1bb None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:22.141728 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a42c18-50cc-44f6-a121-90ec6367b1bb None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:22.142010 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a42c18-50cc-44f6-a121-90ec6367b1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:22.203839 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-85a42c18-50cc-44f6-a121-90ec6367b1bb None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:22.205030 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-85a42c18-50cc-44f6-a121-90ec6367b1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:22.225271 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 52/112] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:29:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:29:22.226607 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb10f8-3fb4-4dfe-bd11-7d7198ace36c None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:22.226907 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb10f8-3fb4-4dfe-bd11-7d7198ace36c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:22.227191 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb10f8-3fb4-4dfe-bd11-7d7198ace36c None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:22.229863 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:22.233456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 53/113] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:22.234294 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb10f8-3fb4-4dfe-bd11-7d7198ace36c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:22.285829 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 61/114] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:22.286699 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a42c18-50cc-44f6-a121-90ec6367b1bb None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:22.365801 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb10f8-3fb4-4dfe-bd11-7d7198ace36c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:22.380856 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:22.402302 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6eee49-1976-482e-b8b0-f2e721665087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:22.403096 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6eee49-1976-482e-b8b0-f2e721665087 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:22.403702 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6eee49-1976-482e-b8b0-f2e721665087 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:22.404071 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6eee49-1976-482e-b8b0-f2e721665087 None admin] PATH_INFO: `/v3/users/11071bdab076477385007310e6ab9852` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:22.406339 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6eee49-1976-482e-b8b0-f2e721665087 None admin] RBAC: Authorizing `identity:get_user(user_id=11071bdab076477385007310e6ab9852)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:22.408051 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6eee49-1976-482e-b8b0-f2e721665087 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:22.408780 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a42c18-50cc-44f6-a121-90ec6367b1bb None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:22.410072 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 54/115] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:22 2022] GET /identity/v3/users/11071bdab076477385007310e6ab9852 => generated 247 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:22.411319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f1554-cd81-48fd-9d42-fb7785a5ccaa None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:22.411726 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f1554-cd81-48fd-9d42-fb7785a5ccaa None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:22.412079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f1554-cd81-48fd-9d42-fb7785a5ccaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:22.430692 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0f1f1554-cd81-48fd-9d42-fb7785a5ccaa None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:22.432147 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0f1f1554-cd81-48fd-9d42-fb7785a5ccaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:22.451635 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d00483-564e-4b6a-8c88-a7b0ed568259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:22.452296 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d00483-564e-4b6a-8c88-a7b0ed568259 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:22.452581 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d00483-564e-4b6a-8c88-a7b0ed568259 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:22.452862 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d00483-564e-4b6a-8c88-a7b0ed568259 None admin] PATH_INFO: `/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:22.454096 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d00483-564e-4b6a-8c88-a7b0ed568259 None admin] RBAC: Authorizing `identity:get_project(project_id=076f7405c1874c1ea0d0f44c11ae4d38)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:22.455626 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d00483-564e-4b6a-8c88-a7b0ed568259 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:22.457441 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 62/116] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:22 2022] GET /identity/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:22.458395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d00483-564e-4b6a-8c88-a7b0ed568259 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:22.488621 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:22.510859 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 55/117] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:22.511562 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f1554-cd81-48fd-9d42-fb7785a5ccaa None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:22.569464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d00483-564e-4b6a-8c88-a7b0ed568259 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:22.606781 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74259478-6819-4a05-a47c-2fbf60654b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:22.607986 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74259478-6819-4a05-a47c-2fbf60654b6b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:22.608275 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74259478-6819-4a05-a47c-2fbf60654b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:22.608651 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74259478-6819-4a05-a47c-2fbf60654b6b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:22.615331 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f1554-cd81-48fd-9d42-fb7785a5ccaa None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:22.619756 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74259478-6819-4a05-a47c-2fbf60654b6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:22.625081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-74259478-6819-4a05-a47c-2fbf60654b6b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:22.625081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:22.625081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:22.625081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:22.625081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:22.625081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:22.625081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:22.625081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:22.625081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:22.625081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:22.625081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:22.625081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:22.625081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:22.625081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:22.625081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:22.625081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:22.627105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:22.627105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:22.627105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:22.627105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:22.627105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:22.627105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:22.627105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:22.627105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:22.627105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:22.627105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:22.627105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:22.627105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:22.627105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:22.627105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:22.627105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:22.627105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:22.629047 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:22.629047 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:22.629047 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:22.629047 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:22.629047 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:22.629047 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:22.629047 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:22.629047 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:22.629047 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:22.629047 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:22.629047 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:22.629047 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:29:22.629047 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 63/118] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:22.629047 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74259478-6819-4a05-a47c-2fbf60654b6b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:22.635435 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:22.656887 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7f2525-37cd-48ee-9740-c86def33948a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:22.657543 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7f2525-37cd-48ee-9740-c86def33948a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:22.657834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7f2525-37cd-48ee-9740-c86def33948a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:22.658465 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7f2525-37cd-48ee-9740-c86def33948a None admin] PATH_INFO: `/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:22.659319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7f2525-37cd-48ee-9740-c86def33948a None admin] RBAC: Authorizing `identity:get_project(project_id=076f7405c1874c1ea0d0f44c11ae4d38)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:22.661428 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7f2525-37cd-48ee-9740-c86def33948a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:22.663235 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 56/119] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:22 2022] GET /identity/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:22.664114 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7f2525-37cd-48ee-9740-c86def33948a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:22.731793 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:22.750160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74259478-6819-4a05-a47c-2fbf60654b6b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:22.774931 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7f2525-37cd-48ee-9740-c86def33948a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:22.790143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23024e8-8006-4c35-86a7-d0c01eb5f510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:22.790772 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23024e8-8006-4c35-86a7-d0c01eb5f510 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:22.791062 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23024e8-8006-4c35-86a7-d0c01eb5f510 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:22.791346 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23024e8-8006-4c35-86a7-d0c01eb5f510 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:22.792231 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23024e8-8006-4c35-86a7-d0c01eb5f510 None admin] RBAC: Adding query filter params (role.id=2b922104c92a4cbc9432fbce1c9bda1e, scope.project.id=076f7405c1874c1ea0d0f44c11ae4d38, user.id=36ceba794e7f467d989738a87a04100c) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:22.792496 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23024e8-8006-4c35-86a7-d0c01eb5f510 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:22.793947 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23024e8-8006-4c35-86a7-d0c01eb5f510 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:22.800205 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 64/120] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:29:22 2022] GET /identity/v3/role_assignments?user.id=36ceba794e7f467d989738a87a04100c&scope.project.id=076f7405c1874c1ea0d0f44c11ae4d38&role.id=2b922104c92a4cbc9432fbce1c9bda1e => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:22.800935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23024e8-8006-4c35-86a7-d0c01eb5f510 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:22.824294 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e39bb-c49f-4647-bb35-2f25409d688d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:22.826081 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e39bb-c49f-4647-bb35-2f25409d688d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:22.826594 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e39bb-c49f-4647-bb35-2f25409d688d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:22.827369 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e39bb-c49f-4647-bb35-2f25409d688d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:22.844278 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e39bb-c49f-4647-bb35-2f25409d688d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:22.851446 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-262e39bb-c49f-4647-bb35-2f25409d688d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:29:22.851446 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:22.851446 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:22.851446 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:22.851446 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:22.851446 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:22.851446 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:22.851446 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:22.851446 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:22.851446 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:22.851446 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:22.851446 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:22.851446 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:22.851446 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:22.851446 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:22.851446 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:22.852336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:22.852336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:22.852336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:22.852336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:22.852336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:22.852336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:22.852336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:22.852336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:22.852336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:22.852336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:22.852336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:22.852336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:22.852336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:22.852336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:22.852336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:22.852336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:22.853102 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:22.853102 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:22.853102 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:22.853102 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:22.853102 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:22.853102 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:22.853102 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:22.853102 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:22.853102 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:22.853102 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:22.853102 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:29:22.853102 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:29:22.853102 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 57/121] 10.222.0.97 () {56 vars in 1114 bytes} [Tue Jul 26 14:29:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:22.854627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff64ba9-731b-4f94-9463-b926e9f66e51 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:22.855256 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff64ba9-731b-4f94-9463-b926e9f66e51 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:22.855655 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff64ba9-731b-4f94-9463-b926e9f66e51 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:22.858448 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 58/122] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:22.859501 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff64ba9-731b-4f94-9463-b926e9f66e51 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:22.925934 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:22.939982 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23024e8-8006-4c35-86a7-d0c01eb5f510 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:22.996208 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c52d48-c930-4734-9ef9-5aeefbf94636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:22.996208 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c52d48-c930-4734-9ef9-5aeefbf94636 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:22.996208 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c52d48-c930-4734-9ef9-5aeefbf94636 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:22.996208 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c52d48-c930-4734-9ef9-5aeefbf94636 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:22.996766 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c52d48-c930-4734-9ef9-5aeefbf94636 None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:22.996766 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c52d48-c930-4734-9ef9-5aeefbf94636 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:22.996766 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c52d48-c930-4734-9ef9-5aeefbf94636 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:23.002390 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff64ba9-731b-4f94-9463-b926e9f66e51 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:23.005916 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 65/123] 10.222.0.97 () {56 vars in 1101 bytes} [Tue Jul 26 14:29:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:23.006750 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c52d48-c930-4734-9ef9-5aeefbf94636 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:23.025017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:23.046763 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd080f63-60f2-49ec-9586-32692171f4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:23.047485 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd080f63-60f2-49ec-9586-32692171f4bd None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:23.047784 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd080f63-60f2-49ec-9586-32692171f4bd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:23.048074 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd080f63-60f2-49ec-9586-32692171f4bd None admin] PATH_INFO: `/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38/users/11071bdab076477385007310e6ab9852/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:23.050307 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd080f63-60f2-49ec-9586-32692171f4bd None admin] RBAC: Authorizing `identity:create_grant(project_id=076f7405c1874c1ea0d0f44c11ae4d38, user_id=11071bdab076477385007310e6ab9852, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:23.053852 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd080f63-60f2-49ec-9586-32692171f4bd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:23.132428 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c52d48-c930-4734-9ef9-5aeefbf94636 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:23.150004 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 59/124] 10.222.0.97 () {58 vars in 1439 bytes} [Tue Jul 26 14:29:22 2022] PUT /identity/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38/users/11071bdab076477385007310e6ab9852/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:29:23.151831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1f884-e719-4752-8eff-e18608e83235 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:23.152734 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1f884-e719-4752-8eff-e18608e83235 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:23.153224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1f884-e719-4752-8eff-e18608e83235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:23.177165 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-9cc1f884-e719-4752-8eff-e18608e83235 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:23.178748 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9cc1f884-e719-4752-8eff-e18608e83235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:23.182933 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d311a4-99d7-4bf9-b10b-20cf177f2d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:23.183608 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d311a4-99d7-4bf9-b10b-20cf177f2d4c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:23.183961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d311a4-99d7-4bf9-b10b-20cf177f2d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:23.184254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d311a4-99d7-4bf9-b10b-20cf177f2d4c None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:23.185049 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d311a4-99d7-4bf9-b10b-20cf177f2d4c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:23.185893 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d311a4-99d7-4bf9-b10b-20cf177f2d4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:23.187313 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d311a4-99d7-4bf9-b10b-20cf177f2d4c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:23.203470 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 66/125] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:29:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:23.204221 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d311a4-99d7-4bf9-b10b-20cf177f2d4c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:23.265894 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 60/126] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:23.267088 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1f884-e719-4752-8eff-e18608e83235 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:23.324837 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:23.343899 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:23.381903 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d311a4-99d7-4bf9-b10b-20cf177f2d4c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:23.449040 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530dc0e6-62c4-437e-8155-69028ed6de9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:23.450104 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dc0e6-62c4-437e-8155-69028ed6de9b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:23.450397 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dc0e6-62c4-437e-8155-69028ed6de9b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:23.450689 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dc0e6-62c4-437e-8155-69028ed6de9b None admin] PATH_INFO: `/v3/users/45da4bf9f6e64bc9bfa6f4e5beafb859` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:23.453450 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530dc0e6-62c4-437e-8155-69028ed6de9b None admin] RBAC: Authorizing `identity:get_user(user_id=45da4bf9f6e64bc9bfa6f4e5beafb859)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:23.454680 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1f884-e719-4752-8eff-e18608e83235 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:23.454997 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530dc0e6-62c4-437e-8155-69028ed6de9b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:23.456997 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 67/127] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:23 2022] GET /identity/v3/users/45da4bf9f6e64bc9bfa6f4e5beafb859 => generated 275 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:23.458699 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530dc0e6-62c4-437e-8155-69028ed6de9b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:23.500614 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221a2ee-bbf9-4130-8d79-bf0889443de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:23.501521 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221a2ee-bbf9-4130-8d79-bf0889443de8 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:23.501807 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221a2ee-bbf9-4130-8d79-bf0889443de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:23.502084 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221a2ee-bbf9-4130-8d79-bf0889443de8 None admin] PATH_INFO: `/v3/users/45da4bf9f6e64bc9bfa6f4e5beafb859` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:23.503538 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3221a2ee-bbf9-4130-8d79-bf0889443de8 None admin] RBAC: Authorizing `identity:get_user(user_id=45da4bf9f6e64bc9bfa6f4e5beafb859)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:23.505931 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3221a2ee-bbf9-4130-8d79-bf0889443de8 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:23.507756 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 61/128] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:23 2022] GET /identity/v3/users/45da4bf9f6e64bc9bfa6f4e5beafb859 => generated 275 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:23.508427 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221a2ee-bbf9-4130-8d79-bf0889443de8 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:23.630238 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530dc0e6-62c4-437e-8155-69028ed6de9b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:23.682872 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8d8ae-ddb0-41e1-a861-731e4d44c959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:23.684569 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8d8ae-ddb0-41e1-a861-731e4d44c959 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:23.684858 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8d8ae-ddb0-41e1-a861-731e4d44c959 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:23.685133 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8d8ae-ddb0-41e1-a861-731e4d44c959 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:23.696398 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8d8ae-ddb0-41e1-a861-731e4d44c959 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:23.701908 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-94b8d8ae-ddb0-41e1-a861-731e4d44c959 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:23.701908 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:23.701908 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:23.701908 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:23.701908 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:23.701908 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:23.701908 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:23.701908 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:23.701908 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:23.701908 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:23.701908 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:23.701908 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:23.701908 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:23.701908 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:23.701908 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:23.701908 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:23.702770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:23.702770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:23.702770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:23.702770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:23.702770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:23.702770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:23.702770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:23.702770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:23.702770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:23.702770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:23.702770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:23.702770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:23.702770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:23.702770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:23.702770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:23.702770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:23.703549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:23.703549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:23.703549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:23.703549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:23.703549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:23.703549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:23.703549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:23.703549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:23.703549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:23.703549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:23.703549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:23.703549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:29:23.703549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 68/129] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:23.704177 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8d8ae-ddb0-41e1-a861-731e4d44c959 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:23.707211 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:23.707336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:23.718173 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221a2ee-bbf9-4130-8d79-bf0889443de8 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:23.784500 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e444029-268e-4acb-93e0-187df86c3f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:23.785962 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e444029-268e-4acb-93e0-187df86c3f5e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:23.786449 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e444029-268e-4acb-93e0-187df86c3f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:23.786925 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e444029-268e-4acb-93e0-187df86c3f5e None admin] PATH_INFO: `/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:23.788339 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e444029-268e-4acb-93e0-187df86c3f5e None admin] RBAC: Authorizing `identity:get_project(project_id=076f7405c1874c1ea0d0f44c11ae4d38)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:23.789998 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e444029-268e-4acb-93e0-187df86c3f5e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:23.792213 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 62/130] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:23 2022] GET /identity/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38 => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:23.793190 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e444029-268e-4acb-93e0-187df86c3f5e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:23.808383 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8d8ae-ddb0-41e1-a861-731e4d44c959 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:23.843673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da4d6d-d2fc-40b8-88d3-5a51a230d285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:23.844590 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da4d6d-d2fc-40b8-88d3-5a51a230d285 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:23.845014 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da4d6d-d2fc-40b8-88d3-5a51a230d285 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:23.845501 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da4d6d-d2fc-40b8-88d3-5a51a230d285 None admin] PATH_INFO: `/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:23.847051 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da4d6d-d2fc-40b8-88d3-5a51a230d285 None admin] RBAC: Authorizing `identity:get_project(project_id=076f7405c1874c1ea0d0f44c11ae4d38)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:23.848514 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da4d6d-d2fc-40b8-88d3-5a51a230d285 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:23.850652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 69/131] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:23 2022] GET /identity/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:23.851567 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da4d6d-d2fc-40b8-88d3-5a51a230d285 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:23.932446 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:23.956735 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e444029-268e-4acb-93e0-187df86c3f5e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:23.975392 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da4d6d-d2fc-40b8-88d3-5a51a230d285 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:24.002229 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6786d-280c-4c40-bbb1-ba8551a9cfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:24.003183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6786d-280c-4c40-bbb1-ba8551a9cfc6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:24.003772 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6786d-280c-4c40-bbb1-ba8551a9cfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:24.004263 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6786d-280c-4c40-bbb1-ba8551a9cfc6 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:24.005195 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6786d-280c-4c40-bbb1-ba8551a9cfc6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:24.005701 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6786d-280c-4c40-bbb1-ba8551a9cfc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:24.007271 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6786d-280c-4c40-bbb1-ba8551a9cfc6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:24.017217 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 63/132] 10.222.0.97 () {56 vars in 1101 bytes} [Tue Jul 26 14:29:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:24.018447 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8e8a3-22cf-4691-a19b-defc36e10944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:24.019385 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6910a9-3840-4b1c-bafc-45a22bf36e4e None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:24.019620 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8e8a3-22cf-4691-a19b-defc36e10944 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:24.019971 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8e8a3-22cf-4691-a19b-defc36e10944 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:24.020293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8e8a3-22cf-4691-a19b-defc36e10944 None admin] PATH_INFO: `/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38/users/45da4bf9f6e64bc9bfa6f4e5beafb859/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:24.020487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6910a9-3840-4b1c-bafc-45a22bf36e4e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:24.020950 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6910a9-3840-4b1c-bafc-45a22bf36e4e None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:24.022930 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 64/133] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:24.026891 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b8e8a3-22cf-4691-a19b-defc36e10944 None admin] RBAC: Authorizing `identity:create_grant(project_id=076f7405c1874c1ea0d0f44c11ae4d38, user_id=45da4bf9f6e64bc9bfa6f4e5beafb859, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:24.026891 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b8e8a3-22cf-4691-a19b-defc36e10944 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:24.028018 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6910a9-3840-4b1c-bafc-45a22bf36e4e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:24.070374 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:24.171752 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6910a9-3840-4b1c-bafc-45a22bf36e4e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:24.202639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 70/134] 10.222.0.97 () {58 vars in 1439 bytes} [Tue Jul 26 14:29:23 2022] PUT /identity/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38/users/45da4bf9f6e64bc9bfa6f4e5beafb859/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:29:24.209178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce98620-1882-4222-a193-c569c6a09793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:24.209890 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce98620-1882-4222-a193-c569c6a09793 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:24.210179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce98620-1882-4222-a193-c569c6a09793 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:24.210467 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce98620-1882-4222-a193-c569c6a09793 None admin] PATH_INFO: `/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38/users/45da4bf9f6e64bc9bfa6f4e5beafb859/roles/0602cf371b0f41b390e8a4499ae6bb3c` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:24.211389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8e8a3-22cf-4691-a19b-defc36e10944 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:24.213044 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce98620-1882-4222-a193-c569c6a09793 None admin] RBAC: Authorizing `identity:create_grant(project_id=076f7405c1874c1ea0d0f44c11ae4d38, user_id=45da4bf9f6e64bc9bfa6f4e5beafb859, role_id=0602cf371b0f41b390e8a4499ae6bb3c)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:24.215107 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce98620-1882-4222-a193-c569c6a09793 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:24.343131 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 65/135] 10.222.0.97 () {58 vars in 1439 bytes} [Tue Jul 26 14:29:24 2022] PUT /identity/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38/users/45da4bf9f6e64bc9bfa6f4e5beafb859/roles/0602cf371b0f41b390e8a4499ae6bb3c => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:29:24.350694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d137739-8c2e-43bf-a0d3-5e9b701658b6 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:24.353213 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d137739-8c2e-43bf-a0d3-5e9b701658b6 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:24.353832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d137739-8c2e-43bf-a0d3-5e9b701658b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:24.394410 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-7d137739-8c2e-43bf-a0d3-5e9b701658b6 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:24.398033 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7d137739-8c2e-43bf-a0d3-5e9b701658b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:24.441640 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8e8a3-22cf-4691-a19b-defc36e10944 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:24.510439 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:24.535616 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c90c09-05e1-451f-aed9-cda6143fabbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:24.536549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c90c09-05e1-451f-aed9-cda6143fabbf None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:24.537462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c90c09-05e1-451f-aed9-cda6143fabbf None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:24.538214 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c90c09-05e1-451f-aed9-cda6143fabbf None admin] PATH_INFO: `/v3/users/45da4bf9f6e64bc9bfa6f4e5beafb859` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:24.540073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c90c09-05e1-451f-aed9-cda6143fabbf None admin] RBAC: Authorizing `identity:get_user(user_id=45da4bf9f6e64bc9bfa6f4e5beafb859)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:24.542104 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c90c09-05e1-451f-aed9-cda6143fabbf None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:24.544873 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 71/136] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:24 2022] GET /identity/v3/users/45da4bf9f6e64bc9bfa6f4e5beafb859 => generated 275 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:24.551165 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c90c09-05e1-451f-aed9-cda6143fabbf None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:24.573104 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 66/137] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:24.611364 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d137739-8c2e-43bf-a0d3-5e9b701658b6 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:24.682488 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:24.688201 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c90c09-05e1-451f-aed9-cda6143fabbf None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:24.740217 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f3cef-a8c6-47c3-a265-bd6241bb9b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:24.741024 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f3cef-a8c6-47c3-a265-bd6241bb9b0b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:24.741372 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f3cef-a8c6-47c3-a265-bd6241bb9b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:24.741652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f3cef-a8c6-47c3-a265-bd6241bb9b0b None admin] PATH_INFO: `/v3/projects/8d0806a2c77d415391f06dc0ed0f21d2` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:24.747475 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f3cef-a8c6-47c3-a265-bd6241bb9b0b None admin] RBAC: Authorizing `identity:get_project(project_id=8d0806a2c77d415391f06dc0ed0f21d2)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:24.747475 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f3cef-a8c6-47c3-a265-bd6241bb9b0b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:24.748351 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 72/138] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:24 2022] GET /identity/v3/projects/8d0806a2c77d415391f06dc0ed0f21d2 => generated 312 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:24.761323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f3cef-a8c6-47c3-a265-bd6241bb9b0b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:24.846206 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d137739-8c2e-43bf-a0d3-5e9b701658b6 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:24.918223 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d4116-436e-4c44-95c4-8c8263056166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:24.918426 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:24.918518 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d4116-436e-4c44-95c4-8c8263056166 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:24.918518 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d4116-436e-4c44-95c4-8c8263056166 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:24.918518 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d4116-436e-4c44-95c4-8c8263056166 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:24.924183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d4116-436e-4c44-95c4-8c8263056166 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:24.929714 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d4116-436e-4c44-95c4-8c8263056166 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:24.929714 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 67/139] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:29:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:24.939424 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d4116-436e-4c44-95c4-8c8263056166 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:24.999471 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f3cef-a8c6-47c3-a265-bd6241bb9b0b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:25.047596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:25.065897 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d4116-436e-4c44-95c4-8c8263056166 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:25.086069 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff985c29-51e6-4ae2-ba9b-bb3f30c913f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:25.086834 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff985c29-51e6-4ae2-ba9b-bb3f30c913f0 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:25.087126 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff985c29-51e6-4ae2-ba9b-bb3f30c913f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:25.087427 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff985c29-51e6-4ae2-ba9b-bb3f30c913f0 None admin] PATH_INFO: `/v3/projects/8d0806a2c77d415391f06dc0ed0f21d2/users/45da4bf9f6e64bc9bfa6f4e5beafb859/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:25.092805 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff985c29-51e6-4ae2-ba9b-bb3f30c913f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d0806a2c77d415391f06dc0ed0f21d2, user_id=45da4bf9f6e64bc9bfa6f4e5beafb859, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:25.094275 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff985c29-51e6-4ae2-ba9b-bb3f30c913f0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:25.108903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7826ac-84ae-4b7e-b7c9-22eafa19198a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:25.109650 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7826ac-84ae-4b7e-b7c9-22eafa19198a None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:25.109942 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7826ac-84ae-4b7e-b7c9-22eafa19198a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:25.110220 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7826ac-84ae-4b7e-b7c9-22eafa19198a None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:25.118201 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7826ac-84ae-4b7e-b7c9-22eafa19198a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:25.118201 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7826ac-84ae-4b7e-b7c9-22eafa19198a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:25.165730 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 73/140] 10.222.0.97 () {58 vars in 1439 bytes} [Tue Jul 26 14:29:24 2022] PUT /identity/v3/projects/8d0806a2c77d415391f06dc0ed0f21d2/users/45da4bf9f6e64bc9bfa6f4e5beafb859/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:29:25.205929 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 68/141] 10.222.0.97 () {60 vars in 1132 bytes} [Tue Jul 26 14:29:24 2022] POST /identity/v3/users => generated 283 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:29:25.375539 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46924eee-4d78-4326-bef2-24d7a238a986 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:25.375539 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46924eee-4d78-4326-bef2-24d7a238a986 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:25.375539 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46924eee-4d78-4326-bef2-24d7a238a986 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:25.377545 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 74/142] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:25.383996 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d34542-741d-4fc2-8d35-8decf02a7636 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:25.384294 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d34542-741d-4fc2-8d35-8decf02a7636 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:25.384568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d34542-741d-4fc2-8d35-8decf02a7636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:25.408936 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-20d34542-741d-4fc2-8d35-8decf02a7636 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:25.410177 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-20d34542-741d-4fc2-8d35-8decf02a7636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:25.506102 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 69/143] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:25.531146 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46924eee-4d78-4326-bef2-24d7a238a986 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:25.648183 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46924eee-4d78-4326-bef2-24d7a238a986 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:25.679622 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:25.691168 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027942b-88f4-4643-bb12-0d1bcd48a23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:25.691859 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027942b-88f4-4643-bb12-0d1bcd48a23a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:25.692144 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027942b-88f4-4643-bb12-0d1bcd48a23a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:25.692424 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027942b-88f4-4643-bb12-0d1bcd48a23a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:25.709833 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a027942b-88f4-4643-bb12-0d1bcd48a23a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:25.714705 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-a027942b-88f4-4643-bb12-0d1bcd48a23a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:29:25.714705 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:25.714705 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:25.714705 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:25.714705 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:25.714705 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:25.714705 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:25.714705 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:25.714705 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:25.714705 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:25.714705 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:25.714705 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:25.714705 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:25.714705 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:25.714705 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:25.714705 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:25.715572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:25.715572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:25.715572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:25.715572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:25.715572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:25.715572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:25.715572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:25.715572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:25.715572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:25.715572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:25.715572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:25.715572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:25.715572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:25.715572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:25.715572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:25.715572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:25.716352 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:25.716352 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:25.716352 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:25.716352 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:25.716352 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:25.716352 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:25.716352 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:25.716352 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:25.716352 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:25.716352 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:25.716352 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:29:25.716352 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:29:25.716352 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 75/144] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:25.723876 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d34542-741d-4fc2-8d35-8decf02a7636 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:25.819920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:25.857151 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d34542-741d-4fc2-8d35-8decf02a7636 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:25.902942 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25805e5f-6dc6-4c8a-bbcc-c2eed3765f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:25.904010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25805e5f-6dc6-4c8a-bbcc-c2eed3765f2c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:25.905006 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25805e5f-6dc6-4c8a-bbcc-c2eed3765f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:25.905514 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25805e5f-6dc6-4c8a-bbcc-c2eed3765f2c None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:25.906495 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25805e5f-6dc6-4c8a-bbcc-c2eed3765f2c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:25.911819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25805e5f-6dc6-4c8a-bbcc-c2eed3765f2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:25.913471 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25805e5f-6dc6-4c8a-bbcc-c2eed3765f2c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:25.922644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 70/145] 10.222.0.97 () {56 vars in 1101 bytes} [Tue Jul 26 14:29:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:25.928600 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027942b-88f4-4643-bb12-0d1bcd48a23a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:26.065959 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027942b-88f4-4643-bb12-0d1bcd48a23a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:26.106745 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788a7af-26d7-4089-a6b0-3bd4255b20ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:26.107520 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788a7af-26d7-4089-a6b0-3bd4255b20ed None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:26.107811 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788a7af-26d7-4089-a6b0-3bd4255b20ed None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:26.108096 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788a7af-26d7-4089-a6b0-3bd4255b20ed None admin] PATH_INFO: `/v3/users/36ceba794e7f467d989738a87a04100c` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:26.112307 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788a7af-26d7-4089-a6b0-3bd4255b20ed None admin] RBAC: Authorizing `identity:get_user(user_id=36ceba794e7f467d989738a87a04100c)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:26.113844 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788a7af-26d7-4089-a6b0-3bd4255b20ed None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:26.115644 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 76/146] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:25 2022] GET /identity/v3/users/36ceba794e7f467d989738a87a04100c => generated 289 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:26.121462 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25805e5f-6dc6-4c8a-bbcc-c2eed3765f2c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:26.168260 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905cffe-2d8b-47eb-a84f-887b485edce7 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:26.168580 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905cffe-2d8b-47eb-a84f-887b485edce7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:26.168872 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905cffe-2d8b-47eb-a84f-887b485edce7 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:26.170778 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 77/147] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:26.178035 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbc637-db56-42a0-a015-0e46576ba0d0 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:26.178331 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbc637-db56-42a0-a015-0e46576ba0d0 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:26.178619 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbc637-db56-42a0-a015-0e46576ba0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:26.204641 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-74cbc637-db56-42a0-a015-0e46576ba0d0 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:26.206144 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-74cbc637-db56-42a0-a015-0e46576ba0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:26.253459 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25805e5f-6dc6-4c8a-bbcc-c2eed3765f2c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:26.272951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:26.292991 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 78/148] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:29:26.296781 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485911ed-f5bb-4c55-902e-23c30381674c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:26.297448 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485911ed-f5bb-4c55-902e-23c30381674c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:26.297740 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485911ed-f5bb-4c55-902e-23c30381674c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:26.298021 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485911ed-f5bb-4c55-902e-23c30381674c None admin] PATH_INFO: `/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:26.299219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485911ed-f5bb-4c55-902e-23c30381674c None admin] RBAC: Authorizing `identity:get_project(project_id=076f7405c1874c1ea0d0f44c11ae4d38)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:26.300715 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485911ed-f5bb-4c55-902e-23c30381674c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:26.302592 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 71/149] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:26 2022] GET /identity/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:26.309335 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cbc637-db56-42a0-a015-0e46576ba0d0 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:26.328364 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485911ed-f5bb-4c55-902e-23c30381674c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:26.332123 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:26.490694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485911ed-f5bb-4c55-902e-23c30381674c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:26.503875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cbc637-db56-42a0-a015-0e46576ba0d0 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:26.552526 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d23a2-042f-4f97-9b4d-3e13f89a548d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:26.553479 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d23a2-042f-4f97-9b4d-3e13f89a548d None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:26.553907 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d23a2-042f-4f97-9b4d-3e13f89a548d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:26.554324 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d23a2-042f-4f97-9b4d-3e13f89a548d None admin] PATH_INFO: `/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38/users/36ceba794e7f467d989738a87a04100c/roles/2b922104c92a4cbc9432fbce1c9bda1e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:26.556604 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d23a2-042f-4f97-9b4d-3e13f89a548d None admin] RBAC: Authorizing `identity:create_grant(project_id=076f7405c1874c1ea0d0f44c11ae4d38, user_id=36ceba794e7f467d989738a87a04100c, role_id=2b922104c92a4cbc9432fbce1c9bda1e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:26.558327 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d23a2-042f-4f97-9b4d-3e13f89a548d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:26.588132 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f235b5-6785-4689-bcb7-393091f1e3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:26.588807 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f235b5-6785-4689-bcb7-393091f1e3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:26.589094 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f235b5-6785-4689-bcb7-393091f1e3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:26.589376 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f235b5-6785-4689-bcb7-393091f1e3d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:26.608603 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f235b5-6785-4689-bcb7-393091f1e3d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:26.614056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-03f235b5-6785-4689-bcb7-393091f1e3d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:29:26.614056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:26.614056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:26.614056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:26.614056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:26.614056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:26.614056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:26.614056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:26.614056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:26.614056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:26.614056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:26.614056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:26.614056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:26.614056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:26.614056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:26.614056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:26.615043 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:26.615043 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:26.615043 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:26.615043 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:26.615043 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:26.615043 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:26.615043 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:26.615043 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:26.615043 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:26.615043 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:26.615043 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:26.615043 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:26.615043 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:26.615043 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:26.615043 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:26.615043 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:26.616319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:26.616319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:26.616319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:26.616319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:26.616319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:26.616319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:26.616319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:26.616319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:26.616319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:26.616319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:26.616319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:29:26.616319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:29:26.616319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 72/150] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:29:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:26.622445 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f235b5-6785-4689-bcb7-393091f1e3d9 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:26.628212 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 79/151] 10.222.0.97 () {58 vars in 1439 bytes} [Tue Jul 26 14:29:26 2022] PUT /identity/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38/users/36ceba794e7f467d989738a87a04100c/roles/2b922104c92a4cbc9432fbce1c9bda1e => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:29:26.710570 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:26.751429 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f235b5-6785-4689-bcb7-393091f1e3d9 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:26.788382 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58390d31-7d2c-49a1-8850-9b9e6e214644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:26.789014 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58390d31-7d2c-49a1-8850-9b9e6e214644 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:26.789300 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58390d31-7d2c-49a1-8850-9b9e6e214644 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:26.789578 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58390d31-7d2c-49a1-8850-9b9e6e214644 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:26.790327 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58390d31-7d2c-49a1-8850-9b9e6e214644 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:26.790619 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58390d31-7d2c-49a1-8850-9b9e6e214644 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:26.792054 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58390d31-7d2c-49a1-8850-9b9e6e214644 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:26.801928 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 73/152] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:26.807420 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d23a2-042f-4f97-9b4d-3e13f89a548d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:26.887599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:26.948060 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d23a2-042f-4f97-9b4d-3e13f89a548d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:26.987604 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a800a-1780-4e2d-a6a3-9da20738d6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:26.988556 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a800a-1780-4e2d-a6a3-9da20738d6da None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:26.988980 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a800a-1780-4e2d-a6a3-9da20738d6da None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:26.989398 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a800a-1780-4e2d-a6a3-9da20738d6da None admin] PATH_INFO: `/v3/users/11071bdab076477385007310e6ab9852` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:26.990768 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a800a-1780-4e2d-a6a3-9da20738d6da None admin] RBAC: Authorizing `identity:get_user(user_id=11071bdab076477385007310e6ab9852)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:26.992421 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a800a-1780-4e2d-a6a3-9da20738d6da None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:26.994393 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 80/153] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:26 2022] GET /identity/v3/users/11071bdab076477385007310e6ab9852 => generated 247 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:27.000686 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58390d31-7d2c-49a1-8850-9b9e6e214644 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:27.122920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58390d31-7d2c-49a1-8850-9b9e6e214644 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:27.151360 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:27.163262 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa42433-7ae5-4ad8-b987-875b679ab0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:27.164173 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa42433-7ae5-4ad8-b987-875b679ab0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:27.164622 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa42433-7ae5-4ad8-b987-875b679ab0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:27.165059 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa42433-7ae5-4ad8-b987-875b679ab0d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:27.246862 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa42433-7ae5-4ad8-b987-875b679ab0d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:27.248732 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa42433-7ae5-4ad8-b987-875b679ab0d2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:27.368482 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 74/154] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:29:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:29:27.373710 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a800a-1780-4e2d-a6a3-9da20738d6da None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:27.489934 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a800a-1780-4e2d-a6a3-9da20738d6da None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:27.509470 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:27.531873 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7695c-3065-4123-8b4e-51e2bb89cbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:27.532574 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7695c-3065-4123-8b4e-51e2bb89cbea None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:27.532950 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7695c-3065-4123-8b4e-51e2bb89cbea None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:27.533234 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7695c-3065-4123-8b4e-51e2bb89cbea None admin] PATH_INFO: `/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:27.540282 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7695c-3065-4123-8b4e-51e2bb89cbea None admin] RBAC: Authorizing `identity:get_project(project_id=076f7405c1874c1ea0d0f44c11ae4d38)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:27.541769 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7695c-3065-4123-8b4e-51e2bb89cbea None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:27.543534 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 81/155] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:27 2022] GET /identity/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:27.550305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa42433-7ae5-4ad8-b987-875b679ab0d2 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:27.580455 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:27.664258 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa42433-7ae5-4ad8-b987-875b679ab0d2 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:27.700246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b174e2-26cf-444c-817b-6f9e25d2d7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:27.700908 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b174e2-26cf-444c-817b-6f9e25d2d7d1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:27.701196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b174e2-26cf-444c-817b-6f9e25d2d7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:27.701475 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b174e2-26cf-444c-817b-6f9e25d2d7d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:27.702285 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b174e2-26cf-444c-817b-6f9e25d2d7d1 None admin] RBAC: Adding query filter params (role.id=51b2d10dfffb4b75b186f3d8b15e2a78, scope.project.id=076f7405c1874c1ea0d0f44c11ae4d38, user.id=11071bdab076477385007310e6ab9852) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:27.702596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b174e2-26cf-444c-817b-6f9e25d2d7d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:27.704158 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b174e2-26cf-444c-817b-6f9e25d2d7d1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:27.714555 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 75/156] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:29:27 2022] GET /identity/v3/role_assignments?user.id=11071bdab076477385007310e6ab9852&scope.project.id=076f7405c1874c1ea0d0f44c11ae4d38&role.id=51b2d10dfffb4b75b186f3d8b15e2a78 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:27.843136 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b44fe4-d3e0-41a4-815b-e66546934038 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:27.843468 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b44fe4-d3e0-41a4-815b-e66546934038 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:27.843724 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b44fe4-d3e0-41a4-815b-e66546934038 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:27.845504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 82/157] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:27.850429 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0221b6-91c6-4daf-8ca4-7424826c0fbd None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:27.850727 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0221b6-91c6-4daf-8ca4-7424826c0fbd None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:27.851012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0221b6-91c6-4daf-8ca4-7424826c0fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:27.876415 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6b0221b6-91c6-4daf-8ca4-7424826c0fbd None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:27.877695 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6b0221b6-91c6-4daf-8ca4-7424826c0fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:27.970979 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 76/158] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:27.983563 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f08b22-4740-4958-b82b-854b66822956 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:27.983856 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f08b22-4740-4958-b82b-854b66822956 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:27.984129 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f08b22-4740-4958-b82b-854b66822956 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:27.985510 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 83/159] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:27.994622 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd09b6c-f774-45c8-a6c7-1156c40cafd1 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:27.994924 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd09b6c-f774-45c8-a6c7-1156c40cafd1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:27.995191 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd09b6c-f774-45c8-a6c7-1156c40cafd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:28.007341 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f08b22-4740-4958-b82b-854b66822956 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:28.039379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-8fd09b6c-f774-45c8-a6c7-1156c40cafd1 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:28.040610 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8fd09b6c-f774-45c8-a6c7-1156c40cafd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:28.073784 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:28.187726 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 77/160] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:28.204194 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f08b22-4740-4958-b82b-854b66822956 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:28.249591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad9457-c7ed-40a7-8dcf-6e505f4910a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:28.250228 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad9457-c7ed-40a7-8dcf-6e505f4910a8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:28.250520 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad9457-c7ed-40a7-8dcf-6e505f4910a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:28.250800 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad9457-c7ed-40a7-8dcf-6e505f4910a8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:28.269612 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd09b6c-f774-45c8-a6c7-1156c40cafd1 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:28.273117 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bad9457-c7ed-40a7-8dcf-6e505f4910a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:28.286519 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-4bad9457-c7ed-40a7-8dcf-6e505f4910a8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:29:28.286519 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:28.286519 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:28.286519 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:28.286519 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:28.286519 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:28.286519 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:28.286519 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:28.286519 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:28.286519 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:28.286519 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:28.286519 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:28.286519 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:28.286519 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:28.286519 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:28.286519 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:28.287546 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:28.287546 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:28.287546 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:28.287546 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:28.287546 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:28.287546 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:28.287546 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:28.287546 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:28.287546 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:28.287546 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:28.287546 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:28.287546 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:28.287546 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:28.287546 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:28.287546 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:28.287546 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:28.288305 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:28.288305 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:28.288305 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:28.288305 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:28.288305 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:28.288305 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:28.288305 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:28.288305 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:28.288305 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:28.288305 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:28.288305 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:29:28.288305 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:29:28.288909 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 84/161] 10.222.0.97 () {56 vars in 1114 bytes} [Tue Jul 26 14:29:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:29:28.307743 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad9457-c7ed-40a7-8dcf-6e505f4910a8 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:28.379748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:28.436057 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd09b6c-f774-45c8-a6c7-1156c40cafd1 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:28.490857 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005e794-da20-44f3-a5a5-788f76618dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:28.491575 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005e794-da20-44f3-a5a5-788f76618dbb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:28.491859 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005e794-da20-44f3-a5a5-788f76618dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:28.492130 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005e794-da20-44f3-a5a5-788f76618dbb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:28.512672 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c005e794-da20-44f3-a5a5-788f76618dbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:28.514763 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad9457-c7ed-40a7-8dcf-6e505f4910a8 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:28.517696 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-c005e794-da20-44f3-a5a5-788f76618dbb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:28.517696 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:28.517696 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:28.517696 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:28.517696 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:28.517696 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:28.517696 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:28.517696 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:28.517696 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:28.517696 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:28.517696 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:28.517696 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:28.517696 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:28.517696 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:28.517696 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:28.517696 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:28.518806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:28.518806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:28.518806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:28.518806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:28.518806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:28.518806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:28.518806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:28.518806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:28.518806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:28.518806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:28.518806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:28.518806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:28.518806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:28.518806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:28.518806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:28.518806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:28.519595 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:28.519595 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:28.519595 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:28.519595 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:28.519595 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:28.519595 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:28.519595 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:28.519595 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:28.519595 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:28.519595 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:28.519595 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:28.519595 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:29:28.523507 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 78/162] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:28.532858 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005e794-da20-44f3-a5a5-788f76618dbb None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:28.551089 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:28.593541 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46474ea7-d045-4e66-a8ac-102f25846f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:28.594275 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46474ea7-d045-4e66-a8ac-102f25846f1c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:28.594467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46474ea7-d045-4e66-a8ac-102f25846f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:28.594747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46474ea7-d045-4e66-a8ac-102f25846f1c None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:28.595576 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46474ea7-d045-4e66-a8ac-102f25846f1c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:28.597540 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46474ea7-d045-4e66-a8ac-102f25846f1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:28.597540 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46474ea7-d045-4e66-a8ac-102f25846f1c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:28.611934 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 85/163] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:29:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:28.617546 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46474ea7-d045-4e66-a8ac-102f25846f1c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:28.711770 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005e794-da20-44f3-a5a5-788f76618dbb None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:28.788193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92055114-c10c-41a0-9a54-3492fc1eb32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:28.788758 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92055114-c10c-41a0-9a54-3492fc1eb32f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:28.789050 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92055114-c10c-41a0-9a54-3492fc1eb32f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:28.789332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92055114-c10c-41a0-9a54-3492fc1eb32f None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:28.790061 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92055114-c10c-41a0-9a54-3492fc1eb32f None admin] RBAC: Adding query filter params (name=member) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:28.790370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92055114-c10c-41a0-9a54-3492fc1eb32f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:28.791791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92055114-c10c-41a0-9a54-3492fc1eb32f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:28.801998 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46474ea7-d045-4e66-a8ac-102f25846f1c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:28.803924 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:28.813491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 79/164] 10.222.0.97 () {56 vars in 1101 bytes} [Tue Jul 26 14:29:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:28.814583 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2a385-e9d7-4b1d-a519-bb0659435f94 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:28.814875 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2a385-e9d7-4b1d-a519-bb0659435f94 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:28.815152 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2a385-e9d7-4b1d-a519-bb0659435f94 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:28.816486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 80/165] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:28.822322 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5a090-00a3-4af7-ad00-06dfed236ea6 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:28.822322 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5a090-00a3-4af7-ad00-06dfed236ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:28.822322 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5a090-00a3-4af7-ad00-06dfed236ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:28.843584 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1d5f0-34e9-4694-b67a-438c80d44dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:28.844246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1d5f0-34e9-4694-b67a-438c80d44dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:28.844537 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1d5f0-34e9-4694-b67a-438c80d44dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:28.844821 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1d5f0-34e9-4694-b67a-438c80d44dc8 None admin] PATH_INFO: `/v3/users/45da4bf9f6e64bc9bfa6f4e5beafb859` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:28.846098 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a1d5f0-34e9-4694-b67a-438c80d44dc8 None admin] RBAC: Authorizing `identity:get_user(user_id=45da4bf9f6e64bc9bfa6f4e5beafb859)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:28.847701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a1d5f0-34e9-4694-b67a-438c80d44dc8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:28.849673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 86/166] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:28 2022] GET /identity/v3/users/45da4bf9f6e64bc9bfa6f4e5beafb859 => generated 275 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:28.850589 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1d5f0-34e9-4694-b67a-438c80d44dc8 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:28.863582 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c6f5a090-00a3-4af7-ad00-06dfed236ea6 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:28.864773 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c6f5a090-00a3-4af7-ad00-06dfed236ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:28.957455 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1d5f0-34e9-4694-b67a-438c80d44dc8 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:28.992715 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e9f34-7f2d-466e-820f-e5ee316cbf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:28.993377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e9f34-7f2d-466e-820f-e5ee316cbf85 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:28.993669 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e9f34-7f2d-466e-820f-e5ee316cbf85 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:28.993961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e9f34-7f2d-466e-820f-e5ee316cbf85 None admin] PATH_INFO: `/v3/users/45da4bf9f6e64bc9bfa6f4e5beafb859` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:28.994057 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 81/167] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:28.995092 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f5a090-00a3-4af7-ad00-06dfed236ea6 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:28.995427 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04e9f34-7f2d-466e-820f-e5ee316cbf85 None admin] RBAC: Authorizing `identity:get_user(user_id=45da4bf9f6e64bc9bfa6f4e5beafb859)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:28.996960 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04e9f34-7f2d-466e-820f-e5ee316cbf85 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:28.999194 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 87/168] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:28 2022] GET /identity/v3/users/45da4bf9f6e64bc9bfa6f4e5beafb859 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:28.999717 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:29.004947 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e9f34-7f2d-466e-820f-e5ee316cbf85 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:29.067908 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:29.113046 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e9f34-7f2d-466e-820f-e5ee316cbf85 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:29.148880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3257b2a-5ab7-44bd-8ba4-2049b5ca7c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:29.149514 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3257b2a-5ab7-44bd-8ba4-2049b5ca7c5e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:29.149799 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3257b2a-5ab7-44bd-8ba4-2049b5ca7c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:29.150084 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3257b2a-5ab7-44bd-8ba4-2049b5ca7c5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:29.207960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f5a090-00a3-4af7-ad00-06dfed236ea6 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:29.225637 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3257b2a-5ab7-44bd-8ba4-2049b5ca7c5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:29.227118 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3257b2a-5ab7-44bd-8ba4-2049b5ca7c5e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:29.267859 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:29.268182 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a776d-c3ef-4b23-adb3-9964890ba9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:29.269026 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a776d-c3ef-4b23-adb3-9964890ba9d4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:29.269308 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a776d-c3ef-4b23-adb3-9964890ba9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:29.269586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a776d-c3ef-4b23-adb3-9964890ba9d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:29.333261 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 88/169] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:29:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 14:29:29.338611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3257b2a-5ab7-44bd-8ba4-2049b5ca7c5e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:29.397062 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:29.399370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574a776d-c3ef-4b23-adb3-9964890ba9d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:29.400876 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574a776d-c3ef-4b23-adb3-9964890ba9d4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:29.446095 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3257b2a-5ab7-44bd-8ba4-2049b5ca7c5e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:29.482802 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84fa21-a414-4ca7-a33d-356514ddd938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:29.483487 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84fa21-a414-4ca7-a33d-356514ddd938 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:29.483877 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84fa21-a414-4ca7-a33d-356514ddd938 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:29.484163 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84fa21-a414-4ca7-a33d-356514ddd938 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:29.496777 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:29.501635 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84fa21-a414-4ca7-a33d-356514ddd938 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:29.510155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-6f84fa21-a414-4ca7-a33d-356514ddd938 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:29.510155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:29.510155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:29.510155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:29.510155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:29.510155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:29.510155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:29.510155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:29.510155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:29.510155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:29.510155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:29.510155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:29.510155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:29.510155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:29.510155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:29.510155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:29.514051 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:29.514051 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:29.514051 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:29.514051 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:29.514051 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:29.514051 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:29.514051 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:29.514051 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:29.514051 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:29.514051 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:29.514051 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:29.514051 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:29.514051 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:29.514051 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:29.514051 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:29.514051 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:29.514823 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:29.514823 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:29.514823 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:29.514823 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:29.514823 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:29.514823 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:29.514823 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:29.514823 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:29.514823 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:29.514823 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:29.514823 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:29.514823 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:29:29.514823 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 89/170] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:29.514823 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84fa21-a414-4ca7-a33d-356514ddd938 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:29.566621 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 82/171] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:29:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:29:29.575566 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a776d-c3ef-4b23-adb3-9964890ba9d4 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:29.633191 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84fa21-a414-4ca7-a33d-356514ddd938 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:29.670354 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e510c-8747-4270-a391-53df0a4162f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:29.670991 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e510c-8747-4270-a391-53df0a4162f8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:29.671325 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e510c-8747-4270-a391-53df0a4162f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:29.671641 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e510c-8747-4270-a391-53df0a4162f8 None admin] PATH_INFO: `/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:29.672965 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e510c-8747-4270-a391-53df0a4162f8 None admin] RBAC: Authorizing `identity:get_project(project_id=076f7405c1874c1ea0d0f44c11ae4d38)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:29.674583 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e510c-8747-4270-a391-53df0a4162f8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:29.676477 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 90/172] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:29 2022] GET /identity/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:29.677170 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e510c-8747-4270-a391-53df0a4162f8 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:29.728528 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:29.731679 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a776d-c3ef-4b23-adb3-9964890ba9d4 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:29.769562 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:29.787255 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e510c-8747-4270-a391-53df0a4162f8 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:29.788233 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf2637-2c1f-4550-863d-3592ace56ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:29.789180 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf2637-2c1f-4550-863d-3592ace56ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:29.789689 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf2637-2c1f-4550-863d-3592ace56ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:29.790156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf2637-2c1f-4550-863d-3592ace56ac1 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:29.791080 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf2637-2c1f-4550-863d-3592ace56ac1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:29.791648 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf2637-2c1f-4550-863d-3592ace56ac1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:29.793305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf2637-2c1f-4550-863d-3592ace56ac1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:29.806503 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 83/173] 10.222.0.97 () {56 vars in 1101 bytes} [Tue Jul 26 14:29:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:29.807642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf2637-2c1f-4550-863d-3592ace56ac1 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:29.825297 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df583d8-414c-499f-9cb4-ac04897d1b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:29.826658 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df583d8-414c-499f-9cb4-ac04897d1b63 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:29.826948 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df583d8-414c-499f-9cb4-ac04897d1b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:29.827228 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df583d8-414c-499f-9cb4-ac04897d1b63 None admin] PATH_INFO: `/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:29.828499 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df583d8-414c-499f-9cb4-ac04897d1b63 None admin] RBAC: Authorizing `identity:get_project(project_id=076f7405c1874c1ea0d0f44c11ae4d38)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:29.830088 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df583d8-414c-499f-9cb4-ac04897d1b63 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:29.832157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 91/174] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:29 2022] GET /identity/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:29.832841 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df583d8-414c-499f-9cb4-ac04897d1b63 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:29.946479 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df583d8-414c-499f-9cb4-ac04897d1b63 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:29.983033 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5de10-c39a-4615-b14a-b61f2055ff41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:29.983783 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5de10-c39a-4615-b14a-b61f2055ff41 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:29.984308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5de10-c39a-4615-b14a-b61f2055ff41 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:29.984684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5de10-c39a-4615-b14a-b61f2055ff41 None admin] PATH_INFO: `/v3/users/45da4bf9f6e64bc9bfa6f4e5beafb859` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:29.985979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae5de10-c39a-4615-b14a-b61f2055ff41 None admin] RBAC: Authorizing `identity:get_user(user_id=45da4bf9f6e64bc9bfa6f4e5beafb859)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:29.987709 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae5de10-c39a-4615-b14a-b61f2055ff41 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:29.989619 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 92/175] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:29 2022] GET /identity/v3/users/45da4bf9f6e64bc9bfa6f4e5beafb859 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:29.990564 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5de10-c39a-4615-b14a-b61f2055ff41 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:29.993488 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:30.006195 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf2637-2c1f-4550-863d-3592ace56ac1 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:30.108424 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5de10-c39a-4615-b14a-b61f2055ff41 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:30.118645 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27529f-aab8-4e4f-ac2d-5dd67d8a6e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:30.119541 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27529f-aab8-4e4f-ac2d-5dd67d8a6e92 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:30.119959 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27529f-aab8-4e4f-ac2d-5dd67d8a6e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:30.120375 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27529f-aab8-4e4f-ac2d-5dd67d8a6e92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:30.121319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c27529f-aab8-4e4f-ac2d-5dd67d8a6e92 None admin] RBAC: Adding query filter params (role.id=e033961a9363456a8510c34414dd839e, scope.project.id=076f7405c1874c1ea0d0f44c11ae4d38, user.id=45da4bf9f6e64bc9bfa6f4e5beafb859) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:30.121787 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c27529f-aab8-4e4f-ac2d-5dd67d8a6e92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:30.124086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c27529f-aab8-4e4f-ac2d-5dd67d8a6e92 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:30.141663 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 84/176] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:29:29 2022] GET /identity/v3/role_assignments?user.id=45da4bf9f6e64bc9bfa6f4e5beafb859&scope.project.id=076f7405c1874c1ea0d0f44c11ae4d38&role.id=e033961a9363456a8510c34414dd839e => generated 608 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:30.142727 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27529f-aab8-4e4f-ac2d-5dd67d8a6e92 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:30.145828 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ec71d-dba2-49dd-b7e3-91438bc5e998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:30.146639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ec71d-dba2-49dd-b7e3-91438bc5e998 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:30.147028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ec71d-dba2-49dd-b7e3-91438bc5e998 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:30.147496 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ec71d-dba2-49dd-b7e3-91438bc5e998 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:30.148454 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ec71d-dba2-49dd-b7e3-91438bc5e998 None admin] RBAC: Adding query filter params (role.id=0602cf371b0f41b390e8a4499ae6bb3c, scope.project.id=076f7405c1874c1ea0d0f44c11ae4d38, user.id=45da4bf9f6e64bc9bfa6f4e5beafb859) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:30.155430 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ec71d-dba2-49dd-b7e3-91438bc5e998 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:30.155430 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ec71d-dba2-49dd-b7e3-91438bc5e998 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:30.157843 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 93/177] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:29:29 2022] GET /identity/v3/role_assignments?user.id=45da4bf9f6e64bc9bfa6f4e5beafb859&scope.project.id=076f7405c1874c1ea0d0f44c11ae4d38&role.id=0602cf371b0f41b390e8a4499ae6bb3c => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:30.159471 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6e598-553d-4d85-9ce9-6ac7473a5856 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:30.159842 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6e598-553d-4d85-9ce9-6ac7473a5856 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:30.160227 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6e598-553d-4d85-9ce9-6ac7473a5856 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:30.161572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 94/178] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:30.168046 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c4a07-39f5-42a0-a834-1e650a5277b5 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:30.168405 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c4a07-39f5-42a0-a834-1e650a5277b5 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:30.168769 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c4a07-39f5-42a0-a834-1e650a5277b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:30.189868 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-010c4a07-39f5-42a0-a834-1e650a5277b5 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:30.191326 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-010c4a07-39f5-42a0-a834-1e650a5277b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:30.217756 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:30.272815 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:30.275710 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 95/179] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:29:30.302088 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c4a07-39f5-42a0-a834-1e650a5277b5 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:30.422017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c4a07-39f5-42a0-a834-1e650a5277b5 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:30.428445 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27529f-aab8-4e4f-ac2d-5dd67d8a6e92 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:30.476930 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a193d-b348-4f73-adb5-7c13113e937c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:30.477592 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a193d-b348-4f73-adb5-7c13113e937c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:30.477949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a193d-b348-4f73-adb5-7c13113e937c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:30.478300 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a193d-b348-4f73-adb5-7c13113e937c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:30.495389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68620b-8aa7-4f2a-a862-69ca6910ebcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:30.496087 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68620b-8aa7-4f2a-a862-69ca6910ebcb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:30.496344 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68620b-8aa7-4f2a-a862-69ca6910ebcb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:30.496619 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68620b-8aa7-4f2a-a862-69ca6910ebcb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:30.524278 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea68620b-8aa7-4f2a-a862-69ca6910ebcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:30.529309 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-ea68620b-8aa7-4f2a-a862-69ca6910ebcb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:29:30.529309 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:30.529309 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:30.529309 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:30.529309 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:30.529309 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:30.529309 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:30.529309 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:30.529309 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:30.529309 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:30.529309 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:30.529309 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:30.529309 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:30.529309 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:30.529309 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:30.529309 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:30.530186 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:30.530186 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:30.530186 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:30.530186 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:30.530186 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:30.530186 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:30.530186 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:30.530186 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:30.530186 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:30.530186 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:30.530186 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:30.530186 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:30.530186 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:30.530186 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:30.530186 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:30.530186 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:30.530955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:30.530955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:30.530955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:30.530955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:30.530955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:30.530955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:30.530955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:30.530955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:30.530955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:30.530955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:30.530955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:29:30.530955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:29:30.530955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 96/180] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:29:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:30.534714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed76ba0-808b-44cc-8987-659898c62bc1 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:30.535005 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed76ba0-808b-44cc-8987-659898c62bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:30.535284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed76ba0-808b-44cc-8987-659898c62bc1 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:30.536641 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 97/181] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:30.542329 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed76ba0-808b-44cc-8987-659898c62bc1 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:30.588050 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84a193d-b348-4f73-adb5-7c13113e937c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:30.589891 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84a193d-b348-4f73-adb5-7c13113e937c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:30.634639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:30.636474 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:30.686928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed76ba0-808b-44cc-8987-659898c62bc1 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:30.717207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 85/182] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:29:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:29:30.718441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db028d-1ebb-42fb-9584-60943c7c775f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:30.718726 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db028d-1ebb-42fb-9584-60943c7c775f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:30.719001 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db028d-1ebb-42fb-9584-60943c7c775f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:30.722690 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28614c-a50b-4525-becf-4a46f6e7cf5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:30.723318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28614c-a50b-4525-becf-4a46f6e7cf5b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:30.723634 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28614c-a50b-4525-becf-4a46f6e7cf5b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:30.723914 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28614c-a50b-4525-becf-4a46f6e7cf5b None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:30.724637 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28614c-a50b-4525-becf-4a46f6e7cf5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:30.724947 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28614c-a50b-4525-becf-4a46f6e7cf5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:30.726324 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28614c-a50b-4525-becf-4a46f6e7cf5b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:30.735705 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 98/183] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:30.736782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffc4cf-4ca7-4663-a1ae-5328154afbcf None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:30.736896 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-70db028d-1ebb-42fb-9584-60943c7c775f None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:30.737075 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffc4cf-4ca7-4663-a1ae-5328154afbcf None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:30.737353 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffc4cf-4ca7-4663-a1ae-5328154afbcf None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:30.738028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-70db028d-1ebb-42fb-9584-60943c7c775f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:30.738693 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 99/184] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:30.740687 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946dc647-f768-41d2-9c45-690e911c9686 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:30.741083 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946dc647-f768-41d2-9c45-690e911c9686 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:30.741366 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946dc647-f768-41d2-9c45-690e911c9686 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:30.742636 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 100/185] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:30.742636 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946dc647-f768-41d2-9c45-690e911c9686 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:30.805108 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 86/186] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:30.805949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db028d-1ebb-42fb-9584-60943c7c775f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:30.851652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946dc647-f768-41d2-9c45-690e911c9686 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:30.861953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:30.868568 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:30.884655 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594f155-c011-494b-8576-6966edccab0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:30.885301 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594f155-c011-494b-8576-6966edccab0c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:30.885587 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594f155-c011-494b-8576-6966edccab0c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:30.885864 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594f155-c011-494b-8576-6966edccab0c None admin] PATH_INFO: `/v3/projects/8d0806a2c77d415391f06dc0ed0f21d2` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:30.887028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8594f155-c011-494b-8576-6966edccab0c None admin] RBAC: Authorizing `identity:get_project(project_id=8d0806a2c77d415391f06dc0ed0f21d2)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:30.889203 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8594f155-c011-494b-8576-6966edccab0c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:30.890807 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 101/187] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:30 2022] GET /identity/v3/projects/8d0806a2c77d415391f06dc0ed0f21d2 => generated 312 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:30.892017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b21f33-00b6-4091-87c3-df584c98ffda None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:30.892312 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b21f33-00b6-4091-87c3-df584c98ffda None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:30.892584 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b21f33-00b6-4091-87c3-df584c98ffda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:30.905645 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db028d-1ebb-42fb-9584-60943c7c775f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:30.910986 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c4b21f33-00b6-4091-87c3-df584c98ffda None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:30.912657 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c4b21f33-00b6-4091-87c3-df584c98ffda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:30.938298 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc719c98-b717-44f9-be0a-58863f7302bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:30.938938 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc719c98-b717-44f9-be0a-58863f7302bb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:30.939220 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc719c98-b717-44f9-be0a-58863f7302bb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:30.939557 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc719c98-b717-44f9-be0a-58863f7302bb None admin] PATH_INFO: `/v3/users/3539a4db36c24bd980bca0a2a943994e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:30.953098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc719c98-b717-44f9-be0a-58863f7302bb None admin] RBAC: Authorizing `identity:get_user(user_id=3539a4db36c24bd980bca0a2a943994e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:30.954545 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc719c98-b717-44f9-be0a-58863f7302bb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:30.956199 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 87/188] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:30 2022] GET /identity/v3/users/3539a4db36c24bd980bca0a2a943994e => generated 283 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:30.961803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a11ecb-72e2-48a4-b8f4-859e53bfdd89 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:30.962090 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a11ecb-72e2-48a4-b8f4-859e53bfdd89 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:30.962375 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a11ecb-72e2-48a4-b8f4-859e53bfdd89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:30.978509 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 102/189] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:30.979370 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b21f33-00b6-4091-87c3-df584c98ffda None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:30.979729 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-14a11ecb-72e2-48a4-b8f4-859e53bfdd89 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:30.980902 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-14a11ecb-72e2-48a4-b8f4-859e53bfdd89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:31.049890 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 88/190] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:31.050740 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a11ecb-72e2-48a4-b8f4-859e53bfdd89 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:31.089687 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b21f33-00b6-4091-87c3-df584c98ffda None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:31.096920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:31.105779 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:31.123881 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d08c05f-96cf-433b-b5d2-fcc7eb084751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:31.124527 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d08c05f-96cf-433b-b5d2-fcc7eb084751 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:31.124808 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d08c05f-96cf-433b-b5d2-fcc7eb084751 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:31.125086 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d08c05f-96cf-433b-b5d2-fcc7eb084751 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:31.134970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d08c05f-96cf-433b-b5d2-fcc7eb084751 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:31.139952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-2d08c05f-96cf-433b-b5d2-fcc7eb084751 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:29:31.139952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:31.139952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:31.139952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:31.139952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:31.139952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:31.139952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:31.139952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:31.139952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:31.139952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:31.139952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:31.139952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:31.139952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:31.139952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:31.139952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:31.139952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:31.140812 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:31.140812 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:31.140812 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:31.140812 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:31.140812 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:31.140812 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:31.140812 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:31.140812 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:31.140812 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:31.140812 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:31.140812 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:31.140812 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:31.140812 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:31.140812 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:31.140812 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:31.140812 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:31.141616 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:31.141616 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:31.141616 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:31.141616 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:31.141616 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:31.141616 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:31.141616 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:31.141616 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:31.141616 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:31.141616 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:31.141616 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:29:31.141616 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:29:31.141616 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 103/191] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:31.142457 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d08c05f-96cf-433b-b5d2-fcc7eb084751 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:31.156214 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a11ecb-72e2-48a4-b8f4-859e53bfdd89 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:31.246666 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee762bf-a1ac-4f1a-96c6-2d3548aa5340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:31.247294 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee762bf-a1ac-4f1a-96c6-2d3548aa5340 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:31.247591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee762bf-a1ac-4f1a-96c6-2d3548aa5340 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:31.247869 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee762bf-a1ac-4f1a-96c6-2d3548aa5340 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:31.248657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee762bf-a1ac-4f1a-96c6-2d3548aa5340 None admin] RBAC: Adding query filter params (role.id=e033961a9363456a8510c34414dd839e, scope.project.id=8d0806a2c77d415391f06dc0ed0f21d2, user.id=45da4bf9f6e64bc9bfa6f4e5beafb859) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:31.248970 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee762bf-a1ac-4f1a-96c6-2d3548aa5340 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:31.250405 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee762bf-a1ac-4f1a-96c6-2d3548aa5340 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:31.256163 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 89/192] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:29:31 2022] GET /identity/v3/role_assignments?user.id=45da4bf9f6e64bc9bfa6f4e5beafb859&scope.project.id=8d0806a2c77d415391f06dc0ed0f21d2&role.id=e033961a9363456a8510c34414dd839e => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:31.256930 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee762bf-a1ac-4f1a-96c6-2d3548aa5340 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:31.338545 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d08c05f-96cf-433b-b5d2-fcc7eb084751 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:31.370723 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee762bf-a1ac-4f1a-96c6-2d3548aa5340 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:31.376661 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26c702-83c0-4ab0-b405-d8a2f2527845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:31.377298 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26c702-83c0-4ab0-b405-d8a2f2527845 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:31.377580 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26c702-83c0-4ab0-b405-d8a2f2527845 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:31.377857 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26c702-83c0-4ab0-b405-d8a2f2527845 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:31.384408 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:31.412996 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270d13fd-e03d-424d-b4e7-d899818e96fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:31.413633 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d13fd-e03d-424d-b4e7-d899818e96fb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:31.413915 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d13fd-e03d-424d-b4e7-d899818e96fb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:31.414193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d13fd-e03d-424d-b4e7-d899818e96fb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:31.431458 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270d13fd-e03d-424d-b4e7-d899818e96fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:31.438313 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-270d13fd-e03d-424d-b4e7-d899818e96fb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:29:31.438313 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:31.438313 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:31.438313 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:31.438313 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:31.438313 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:31.438313 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:31.438313 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:31.438313 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:31.438313 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:31.438313 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:31.438313 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:31.438313 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:31.438313 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:31.438313 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:31.438313 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:31.439176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:31.439176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:31.439176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:31.439176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:31.439176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:31.439176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:31.439176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:31.439176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:31.439176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:31.439176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:31.439176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:31.439176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:31.439176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:31.439176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:31.439176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:31.439176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:31.439986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:31.439986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:31.439986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:31.439986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:31.439986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:31.439986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:31.439986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:31.439986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:31.439986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:31.439986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:31.439986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:29:31.439986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:29:31.439986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 90/193] 10.222.0.97 () {56 vars in 1114 bytes} [Tue Jul 26 14:29:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:31.442419 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270d13fd-e03d-424d-b4e7-d899818e96fb None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:31.442832 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:31.470966 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26c702-83c0-4ab0-b405-d8a2f2527845 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:31.473253 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26c702-83c0-4ab0-b405-d8a2f2527845 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:31.547540 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270d13fd-e03d-424d-b4e7-d899818e96fb None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:31.572640 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 104/194] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:29:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:29:31.573478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26c702-83c0-4ab0-b405-d8a2f2527845 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:31.582866 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b258477d-f2cf-43ac-8d95-40843b3007f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:31.583619 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258477d-f2cf-43ac-8d95-40843b3007f1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:31.583903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258477d-f2cf-43ac-8d95-40843b3007f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:31.584180 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258477d-f2cf-43ac-8d95-40843b3007f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:31.585353 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b258477d-f2cf-43ac-8d95-40843b3007f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:31.586804 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b258477d-f2cf-43ac-8d95-40843b3007f1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:31.588485 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 91/195] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:29:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:31.589286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b258477d-f2cf-43ac-8d95-40843b3007f1 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:31.624973 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:31.648859 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:31.671036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26c702-83c0-4ab0-b405-d8a2f2527845 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:31.688019 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b258477d-f2cf-43ac-8d95-40843b3007f1 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:31.705534 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47cee5e-bba1-46e4-9d88-23ba00c98896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:31.706148 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cee5e-bba1-46e4-9d88-23ba00c98896 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:31.706430 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cee5e-bba1-46e4-9d88-23ba00c98896 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:31.706705 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cee5e-bba1-46e4-9d88-23ba00c98896 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:31.707470 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47cee5e-bba1-46e4-9d88-23ba00c98896 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:31.707782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47cee5e-bba1-46e4-9d88-23ba00c98896 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:31.709168 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47cee5e-bba1-46e4-9d88-23ba00c98896 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:31.717923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 105/196] 10.222.0.97 () {56 vars in 1101 bytes} [Tue Jul 26 14:29:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:31.718655 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47cee5e-bba1-46e4-9d88-23ba00c98896 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:31.723690 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f3bb3-7729-4e0d-b847-c81d6bcbedd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:31.724366 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f3bb3-7729-4e0d-b847-c81d6bcbedd7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:31.724650 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f3bb3-7729-4e0d-b847-c81d6bcbedd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:31.724932 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f3bb3-7729-4e0d-b847-c81d6bcbedd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:31.725844 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f3bb3-7729-4e0d-b847-c81d6bcbedd7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:31.726156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f3bb3-7729-4e0d-b847-c81d6bcbedd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:31.727593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f3bb3-7729-4e0d-b847-c81d6bcbedd7 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:31.735913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 92/197] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:29:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:31.737942 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f3bb3-7729-4e0d-b847-c81d6bcbedd7 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:31.818744 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47cee5e-bba1-46e4-9d88-23ba00c98896 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:31.831955 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:31.835606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f3bb3-7729-4e0d-b847-c81d6bcbedd7 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:31.851829 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd9375f-c53a-4735-af8c-b6bebc5afe9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:31.852406 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9375f-c53a-4735-af8c-b6bebc5afe9d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:31.852692 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9375f-c53a-4735-af8c-b6bebc5afe9d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:31.852970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9375f-c53a-4735-af8c-b6bebc5afe9d None admin] PATH_INFO: `/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:31.854217 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd9375f-c53a-4735-af8c-b6bebc5afe9d None admin] RBAC: Authorizing `identity:get_project(project_id=5dbf1af61d63414cbf4c33bdc412a9d6)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:31.855651 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd9375f-c53a-4735-af8c-b6bebc5afe9d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:31.858044 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 106/198] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:31 2022] GET /identity/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:31.858620 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd9375f-c53a-4735-af8c-b6bebc5afe9d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:31.867511 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:31.869559 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87fcc25-6e80-4d3d-afbb-eccb9137420e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:31.870176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87fcc25-6e80-4d3d-afbb-eccb9137420e None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:31.870460 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87fcc25-6e80-4d3d-afbb-eccb9137420e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:31.870741 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87fcc25-6e80-4d3d-afbb-eccb9137420e None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:31.871747 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87fcc25-6e80-4d3d-afbb-eccb9137420e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:31.874040 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87fcc25-6e80-4d3d-afbb-eccb9137420e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:31.957384 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd9375f-c53a-4735-af8c-b6bebc5afe9d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:31.990238 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7fc79-0315-49ce-9e63-3162c7122236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:31.990875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7fc79-0315-49ce-9e63-3162c7122236 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:31.991160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7fc79-0315-49ce-9e63-3162c7122236 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:31.991460 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7fc79-0315-49ce-9e63-3162c7122236 None admin] PATH_INFO: `/v3/users/36ceba794e7f467d989738a87a04100c` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:31.992628 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce7fc79-0315-49ce-9e63-3162c7122236 None admin] RBAC: Authorizing `identity:get_user(user_id=36ceba794e7f467d989738a87a04100c)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:31.994069 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce7fc79-0315-49ce-9e63-3162c7122236 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:31.995708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 107/199] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:31 2022] GET /identity/v3/users/36ceba794e7f467d989738a87a04100c => generated 289 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:31.996473 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7fc79-0315-49ce-9e63-3162c7122236 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:32.007448 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 93/200] 10.222.0.97 () {60 vars in 1132 bytes} [Tue Jul 26 14:29:31 2022] POST /identity/v3/users => generated 297 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:29:32.008392 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87fcc25-6e80-4d3d-afbb-eccb9137420e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:32.090204 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:32.117729 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7fc79-0315-49ce-9e63-3162c7122236 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:32.131001 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87fcc25-6e80-4d3d-afbb-eccb9137420e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:32.155719 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee8282-f160-45fd-ad35-5ea5c7a3fd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:32.156375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee8282-f160-45fd-ad35-5ea5c7a3fd20 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:32.156660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee8282-f160-45fd-ad35-5ea5c7a3fd20 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:32.156940 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee8282-f160-45fd-ad35-5ea5c7a3fd20 None admin] PATH_INFO: `/v3/users/3539a4db36c24bd980bca0a2a943994e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:32.158759 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dee8282-f160-45fd-ad35-5ea5c7a3fd20 None admin] RBAC: Authorizing `identity:get_user(user_id=3539a4db36c24bd980bca0a2a943994e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:32.159754 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dee8282-f160-45fd-ad35-5ea5c7a3fd20 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:32.161406 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 108/201] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:31 2022] GET /identity/v3/users/3539a4db36c24bd980bca0a2a943994e => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:32.161406 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee8282-f160-45fd-ad35-5ea5c7a3fd20 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:32.175157 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f87407-09eb-408f-ba40-905b0e2820fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:32.175880 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f87407-09eb-408f-ba40-905b0e2820fd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:32.176158 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f87407-09eb-408f-ba40-905b0e2820fd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:32.176438 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f87407-09eb-408f-ba40-905b0e2820fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:32.177235 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f87407-09eb-408f-ba40-905b0e2820fd None admin] RBAC: Adding query filter params (role.id=51b2d10dfffb4b75b186f3d8b15e2a78, scope.project.id=5dbf1af61d63414cbf4c33bdc412a9d6, user.id=3539a4db36c24bd980bca0a2a943994e) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:32.177549 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f87407-09eb-408f-ba40-905b0e2820fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:32.179463 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f87407-09eb-408f-ba40-905b0e2820fd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:32.187868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 94/202] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:29:32 2022] GET /identity/v3/role_assignments?user.id=3539a4db36c24bd980bca0a2a943994e&scope.project.id=5dbf1af61d63414cbf4c33bdc412a9d6&role.id=51b2d10dfffb4b75b186f3d8b15e2a78 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:32.188016 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f87407-09eb-408f-ba40-905b0e2820fd None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:32.214503 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:32.292358 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee8282-f160-45fd-ad35-5ea5c7a3fd20 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:32.345307 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f87407-09eb-408f-ba40-905b0e2820fd None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:32.345825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f74f6f3-26f5-4274-adc1-d32c611faaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:32.347794 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74f6f3-26f5-4274-adc1-d32c611faaed None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:32.347794 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74f6f3-26f5-4274-adc1-d32c611faaed None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:32.347794 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74f6f3-26f5-4274-adc1-d32c611faaed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:32.361566 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:32.387678 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397696f3-47cf-4190-92f7-09abce5b9bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:32.388942 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397696f3-47cf-4190-92f7-09abce5b9bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:32.389187 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397696f3-47cf-4190-92f7-09abce5b9bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:32.389473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397696f3-47cf-4190-92f7-09abce5b9bf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:32.447792 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f74f6f3-26f5-4274-adc1-d32c611faaed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:32.450521 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f74f6f3-26f5-4274-adc1-d32c611faaed None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:32.477865 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397696f3-47cf-4190-92f7-09abce5b9bf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:32.479593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397696f3-47cf-4190-92f7-09abce5b9bf7 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:32.499888 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:32.585955 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 95/203] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:29:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:29:32.587960 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 109/204] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:29:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:29:32.595375 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397696f3-47cf-4190-92f7-09abce5b9bf7 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:32.595568 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f74f6f3-26f5-4274-adc1-d32c611faaed None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:32.678028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:32.751925 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397696f3-47cf-4190-92f7-09abce5b9bf7 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:32.775708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f74f6f3-26f5-4274-adc1-d32c611faaed None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:32.783665 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:32.817182 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af816450-6b62-4da5-ae60-1ada5f0544f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:32.817830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af816450-6b62-4da5-ae60-1ada5f0544f3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:32.818111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af816450-6b62-4da5-ae60-1ada5f0544f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:32.818388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af816450-6b62-4da5-ae60-1ada5f0544f3 None admin] PATH_INFO: `/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:32.819946 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af816450-6b62-4da5-ae60-1ada5f0544f3 None admin] RBAC: Authorizing `identity:get_project(project_id=5dbf1af61d63414cbf4c33bdc412a9d6)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:32.821396 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af816450-6b62-4da5-ae60-1ada5f0544f3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:32.823068 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 96/205] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:32 2022] GET /identity/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6 => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:32.828997 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af816450-6b62-4da5-ae60-1ada5f0544f3 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:32.841611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5dc740-2ee1-4d2f-9cab-0c86eac55428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:32.842270 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5dc740-2ee1-4d2f-9cab-0c86eac55428 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:32.842550 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5dc740-2ee1-4d2f-9cab-0c86eac55428 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:32.842820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5dc740-2ee1-4d2f-9cab-0c86eac55428 None admin] PATH_INFO: `/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:32.845632 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5dc740-2ee1-4d2f-9cab-0c86eac55428 None admin] RBAC: Authorizing `identity:get_project(project_id=076f7405c1874c1ea0d0f44c11ae4d38)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:32.847100 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5dc740-2ee1-4d2f-9cab-0c86eac55428 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:32.850382 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 110/206] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:32 2022] GET /identity/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38 => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:32.856808 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5dc740-2ee1-4d2f-9cab-0c86eac55428 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:32.992963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af816450-6b62-4da5-ae60-1ada5f0544f3 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:33.031706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05b7f7-5002-4132-8492-064cf80b8b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:33.032814 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05b7f7-5002-4132-8492-064cf80b8b66 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:33.033102 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05b7f7-5002-4132-8492-064cf80b8b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:33.033654 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05b7f7-5002-4132-8492-064cf80b8b66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:33.034468 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05b7f7-5002-4132-8492-064cf80b8b66 None admin] RBAC: Adding query filter params (role.id=0602cf371b0f41b390e8a4499ae6bb3c, scope.project.id=5dbf1af61d63414cbf4c33bdc412a9d6, user.id=3539a4db36c24bd980bca0a2a943994e) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:33.035033 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05b7f7-5002-4132-8492-064cf80b8b66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:33.036587 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05b7f7-5002-4132-8492-064cf80b8b66 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:33.047106 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 97/207] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:29:32 2022] GET /identity/v3/role_assignments?user.id=3539a4db36c24bd980bca0a2a943994e&scope.project.id=5dbf1af61d63414cbf4c33bdc412a9d6&role.id=0602cf371b0f41b390e8a4499ae6bb3c => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:33.057867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5dc740-2ee1-4d2f-9cab-0c86eac55428 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:33.071276 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:33.103723 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251976b3-d40d-45c5-97eb-eb2b4eb952ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:33.104602 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251976b3-d40d-45c5-97eb-eb2b4eb952ba None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:33.105031 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251976b3-d40d-45c5-97eb-eb2b4eb952ba None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:33.105450 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251976b3-d40d-45c5-97eb-eb2b4eb952ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:33.106395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251976b3-d40d-45c5-97eb-eb2b4eb952ba None admin] RBAC: Adding query filter params (role.id=2b922104c92a4cbc9432fbce1c9bda1e, scope.project.id=076f7405c1874c1ea0d0f44c11ae4d38, user.id=36ceba794e7f467d989738a87a04100c) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:33.107427 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251976b3-d40d-45c5-97eb-eb2b4eb952ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:33.109028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251976b3-d40d-45c5-97eb-eb2b4eb952ba None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:33.116168 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 111/208] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:29:32 2022] GET /identity/v3/role_assignments?user.id=36ceba794e7f467d989738a87a04100c&scope.project.id=076f7405c1874c1ea0d0f44c11ae4d38&role.id=2b922104c92a4cbc9432fbce1c9bda1e => generated 608 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:34.425518 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909046c-8c1c-43f9-9105-b34f647d670a None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:34.426024 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909046c-8c1c-43f9-9105-b34f647d670a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:34.426024 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909046c-8c1c-43f9-9105-b34f647d670a None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:34.427346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 98/209] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:34.432573 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9008f-d562-4aed-bb06-5cb00d09860b None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:34.432970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9008f-d562-4aed-bb06-5cb00d09860b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:34.433156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9008f-d562-4aed-bb06-5cb00d09860b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:34.455427 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c6a9008f-d562-4aed-bb06-5cb00d09860b None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:34.456913 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c6a9008f-d562-4aed-bb06-5cb00d09860b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:34.474043 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec5f38-a560-4461-bf02-bbe75981a541 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:34.474644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec5f38-a560-4461-bf02-bbe75981a541 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:34.475145 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec5f38-a560-4461-bf02-bbe75981a541 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:34.476730 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 99/210] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:29:34.484439 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d4d5b-7552-46b0-8b6c-e02e442b99bc None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:34.484973 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d4d5b-7552-46b0-8b6c-e02e442b99bc None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:34.485473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d4d5b-7552-46b0-8b6c-e02e442b99bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:34.504107 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-560d4d5b-7552-46b0-8b6c-e02e442b99bc None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:34.505634 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-560d4d5b-7552-46b0-8b6c-e02e442b99bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:34.539688 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 112/211] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:34.539688 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f521fb-d081-4bb4-9050-c5bd9623f400 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:34.539688 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f521fb-d081-4bb4-9050-c5bd9623f400 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:34.539688 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f521fb-d081-4bb4-9050-c5bd9623f400 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:34.539688 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 113/212] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:34.545018 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb94afd-230c-4c99-ba80-90240c2777c0 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:34.546027 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb94afd-230c-4c99-ba80-90240c2777c0 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:34.546897 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb94afd-230c-4c99-ba80-90240c2777c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:34.563833 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:34.570204 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9eb94afd-230c-4c99-ba80-90240c2777c0 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:34.571825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9eb94afd-230c-4c99-ba80-90240c2777c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:34.584698 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 100/213] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:29:34.585355 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d4d5b-7552-46b0-8b6c-e02e442b99bc None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:34.650781 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 114/214] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:29:34.656433 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb94afd-230c-4c99-ba80-90240c2777c0 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:34.698425 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:34.704430 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d4d5b-7552-46b0-8b6c-e02e442b99bc None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:34.740162 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf12a62-58ed-4a11-b88d-309ab40ce0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:34.740814 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf12a62-58ed-4a11-b88d-309ab40ce0c8 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:34.741102 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf12a62-58ed-4a11-b88d-309ab40ce0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:34.741385 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf12a62-58ed-4a11-b88d-309ab40ce0c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:34.752295 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf12a62-58ed-4a11-b88d-309ab40ce0c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:34.758508 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-fcf12a62-58ed-4a11-b88d-309ab40ce0c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:29:34.758508 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:34.758508 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:34.758508 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:34.758508 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:34.758508 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:34.758508 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:34.758508 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:34.758508 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:34.758508 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:34.758508 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:34.758508 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:34.758508 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:34.758508 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:34.758508 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:34.758508 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:34.759370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:34.759370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:34.759370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:34.759370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:34.759370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:34.759370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:34.759370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:34.759370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:34.759370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:34.759370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:34.759370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:34.759370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:34.759370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:34.759370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:34.759370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:34.759370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:34.760179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:34.760179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:34.760179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:34.760179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:34.760179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:34.760179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:34.760179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:34.760179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:34.760179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:34.760179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:34.760179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:29:34.760179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:29:34.760775 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 101/215] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:29:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:34.762271 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf12a62-58ed-4a11-b88d-309ab40ce0c8 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:34.771742 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb94afd-230c-4c99-ba80-90240c2777c0 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:34.807211 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:34.816221 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1859d-6c03-4980-acd8-fa8cb3e0173c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:34.817047 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1859d-6c03-4980-acd8-fa8cb3e0173c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:34.819448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1859d-6c03-4980-acd8-fa8cb3e0173c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:34.819448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1859d-6c03-4980-acd8-fa8cb3e0173c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:34.829413 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c1859d-6c03-4980-acd8-fa8cb3e0173c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:34.834395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-92c1859d-6c03-4980-acd8-fa8cb3e0173c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:34.834395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:34.834395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:34.834395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:34.834395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:34.834395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:34.834395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:34.834395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:34.834395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:34.834395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:34.834395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:34.834395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:34.834395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:34.834395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:34.834395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:34.834395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:34.835241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:34.835241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:34.835241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:34.835241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:34.835241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:34.835241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:34.835241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:34.835241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:34.835241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:34.835241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:34.835241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:34.835241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:34.835241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:34.835241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:34.835241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:34.835241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:34.841336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:34.841336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:34.841336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:34.841336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:34.841336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:34.841336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:34.841336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:34.841336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:34.841336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:34.841336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:34.841336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:34.841336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:29:34.841336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 115/216] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:34.841336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1859d-6c03-4980-acd8-fa8cb3e0173c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:34.883444 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf12a62-58ed-4a11-b88d-309ab40ce0c8 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:34.921501 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf8483-76b4-49d6-9a10-03065b7e812f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:34.922551 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf8483-76b4-49d6-9a10-03065b7e812f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:34.922838 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf8483-76b4-49d6-9a10-03065b7e812f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:34.923114 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf8483-76b4-49d6-9a10-03065b7e812f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:34.924664 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bf8483-76b4-49d6-9a10-03065b7e812f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:34.926468 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bf8483-76b4-49d6-9a10-03065b7e812f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:34.932583 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 102/217] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:29:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:34.932583 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf8483-76b4-49d6-9a10-03065b7e812f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:34.947873 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:34.956297 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1859d-6c03-4980-acd8-fa8cb3e0173c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:34.998854 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006e820-d834-44a7-a1cf-867805d3cf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:35.000025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006e820-d834-44a7-a1cf-867805d3cf95 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:35.000544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006e820-d834-44a7-a1cf-867805d3cf95 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:35.001078 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006e820-d834-44a7-a1cf-867805d3cf95 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:35.001973 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006e820-d834-44a7-a1cf-867805d3cf95 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:35.002454 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006e820-d834-44a7-a1cf-867805d3cf95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:35.004136 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006e820-d834-44a7-a1cf-867805d3cf95 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:35.013651 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 116/218] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:35.014661 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006e820-d834-44a7-a1cf-867805d3cf95 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:35.046265 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf8483-76b4-49d6-9a10-03065b7e812f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:35.062264 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:35.087435 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2db06-e7f6-4ede-902a-2b76626207cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:35.088053 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2db06-e7f6-4ede-902a-2b76626207cc None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:35.088343 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2db06-e7f6-4ede-902a-2b76626207cc None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:35.088619 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2db06-e7f6-4ede-902a-2b76626207cc None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:35.089340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c2db06-e7f6-4ede-902a-2b76626207cc None admin] RBAC: Adding query filter params (name=member) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:35.089648 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c2db06-e7f6-4ede-902a-2b76626207cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:35.091076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c2db06-e7f6-4ede-902a-2b76626207cc None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:35.099577 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 103/219] 10.222.0.97 () {56 vars in 1101 bytes} [Tue Jul 26 14:29:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:35.101274 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2db06-e7f6-4ede-902a-2b76626207cc None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:35.134082 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006e820-d834-44a7-a1cf-867805d3cf95 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:35.177280 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e91393-bf57-432f-9da3-808d0f9c75c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:35.178329 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e91393-bf57-432f-9da3-808d0f9c75c6 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:35.178820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e91393-bf57-432f-9da3-808d0f9c75c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:35.179295 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e91393-bf57-432f-9da3-808d0f9c75c6 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:35.180209 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e91393-bf57-432f-9da3-808d0f9c75c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:35.182344 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e91393-bf57-432f-9da3-808d0f9c75c6 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:35.207261 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2db06-e7f6-4ede-902a-2b76626207cc None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:35.240557 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6031d5-7af9-4a83-8c1b-11a136e57ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:35.241194 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6031d5-7af9-4a83-8c1b-11a136e57ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:35.241481 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6031d5-7af9-4a83-8c1b-11a136e57ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:35.241755 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6031d5-7af9-4a83-8c1b-11a136e57ea6 None admin] PATH_INFO: `/v3/users/3539a4db36c24bd980bca0a2a943994e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:35.243124 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6031d5-7af9-4a83-8c1b-11a136e57ea6 None admin] RBAC: Authorizing `identity:get_user(user_id=3539a4db36c24bd980bca0a2a943994e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:35.244600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6031d5-7af9-4a83-8c1b-11a136e57ea6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:35.246188 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 104/220] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:35 2022] GET /identity/v3/users/3539a4db36c24bd980bca0a2a943994e => generated 283 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:35.247470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6031d5-7af9-4a83-8c1b-11a136e57ea6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:35.287394 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:35.347671 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6031d5-7af9-4a83-8c1b-11a136e57ea6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:35.380161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e40df6-c9c8-4d23-baff-f0736f39215e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:35.380800 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e40df6-c9c8-4d23-baff-f0736f39215e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:35.381080 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e40df6-c9c8-4d23-baff-f0736f39215e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:35.381351 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e40df6-c9c8-4d23-baff-f0736f39215e None admin] PATH_INFO: `/v3/users/03f24986305c4ed2913113729807ebec` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:35.395570 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e40df6-c9c8-4d23-baff-f0736f39215e None admin] RBAC: Authorizing `identity:get_user(user_id=03f24986305c4ed2913113729807ebec)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:35.397067 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e40df6-c9c8-4d23-baff-f0736f39215e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:35.398684 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 105/221] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:35 2022] GET /identity/v3/users/03f24986305c4ed2913113729807ebec => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:35.399834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb93d7-4ad9-4ea8-b6e1-68237567b2b1 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:35.400117 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb93d7-4ad9-4ea8-b6e1-68237567b2b1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:35.400388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb93d7-4ad9-4ea8-b6e1-68237567b2b1 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:35.401593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 106/222] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:35.402278 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb93d7-4ad9-4ea8-b6e1-68237567b2b1 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:35.439316 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 117/223] 10.222.0.97 () {60 vars in 1132 bytes} [Tue Jul 26 14:29:35 2022] POST /identity/v3/users => generated 297 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:29:35.439316 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e91393-bf57-432f-9da3-808d0f9c75c6 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:35.465581 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:35.511805 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:35.519765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb93d7-4ad9-4ea8-b6e1-68237567b2b1 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:35.555837 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb08bdd0-2bf6-48e4-8d26-0b4ccfa4b842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:35.556517 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08bdd0-2bf6-48e4-8d26-0b4ccfa4b842 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:35.556800 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08bdd0-2bf6-48e4-8d26-0b4ccfa4b842 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:35.557078 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08bdd0-2bf6-48e4-8d26-0b4ccfa4b842 None admin] PATH_INFO: `/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:35.558268 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb08bdd0-2bf6-48e4-8d26-0b4ccfa4b842 None admin] RBAC: Authorizing `identity:get_project(project_id=5dbf1af61d63414cbf4c33bdc412a9d6)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:35.560319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb08bdd0-2bf6-48e4-8d26-0b4ccfa4b842 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:35.566748 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 107/224] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:35 2022] GET /identity/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:35.566938 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e91393-bf57-432f-9da3-808d0f9c75c6 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:35.567028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79842acf-91a0-4d2c-ae2d-1de376bea0b6 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:35.567028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79842acf-91a0-4d2c-ae2d-1de376bea0b6 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:35.567028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79842acf-91a0-4d2c-ae2d-1de376bea0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:35.583329 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-79842acf-91a0-4d2c-ae2d-1de376bea0b6 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:35.584620 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-79842acf-91a0-4d2c-ae2d-1de376bea0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:35.605829 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba53ee-0c2b-4093-bf12-f1888b875c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:35.606988 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba53ee-0c2b-4093-bf12-f1888b875c1d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:35.607346 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba53ee-0c2b-4093-bf12-f1888b875c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:35.607682 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba53ee-0c2b-4093-bf12-f1888b875c1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:35.668903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 108/225] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:35.669992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79842acf-91a0-4d2c-ae2d-1de376bea0b6 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:35.704737 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ba53ee-0c2b-4093-bf12-f1888b875c1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:35.706244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ba53ee-0c2b-4093-bf12-f1888b875c1d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:35.726650 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:35.795338 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:35.803501 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79842acf-91a0-4d2c-ae2d-1de376bea0b6 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:35.831736 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 118/226] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:29:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:29:35.832607 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba53ee-0c2b-4093-bf12-f1888b875c1d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:35.846156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971327d6-2e5b-4afd-b9e8-f419d9686c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:35.846984 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971327d6-2e5b-4afd-b9e8-f419d9686c5e None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:35.847364 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971327d6-2e5b-4afd-b9e8-f419d9686c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:35.847676 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971327d6-2e5b-4afd-b9e8-f419d9686c5e None admin] PATH_INFO: `/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6/users/3539a4db36c24bd980bca0a2a943994e/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:35.849768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971327d6-2e5b-4afd-b9e8-f419d9686c5e None admin] RBAC: Authorizing `identity:create_grant(project_id=5dbf1af61d63414cbf4c33bdc412a9d6, user_id=3539a4db36c24bd980bca0a2a943994e, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:35.851338 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971327d6-2e5b-4afd-b9e8-f419d9686c5e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:35.920794 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 109/227] 10.222.0.97 () {58 vars in 1439 bytes} [Tue Jul 26 14:29:35 2022] PUT /identity/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6/users/3539a4db36c24bd980bca0a2a943994e/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:29:35.924208 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971327d6-2e5b-4afd-b9e8-f419d9686c5e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:35.973627 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba53ee-0c2b-4093-bf12-f1888b875c1d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:35.998949 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:36.045173 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799845c-4329-46c0-9da3-04d24368d5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:36.045848 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799845c-4329-46c0-9da3-04d24368d5f5 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:36.046131 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799845c-4329-46c0-9da3-04d24368d5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:36.046408 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799845c-4329-46c0-9da3-04d24368d5f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:36.078890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8799845c-4329-46c0-9da3-04d24368d5f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:36.084504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-8799845c-4329-46c0-9da3-04d24368d5f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:29:36.084504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:36.084504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:36.084504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:36.084504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:36.084504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:36.084504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:36.084504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:36.084504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:36.084504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:36.084504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:36.084504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:36.084504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:36.084504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:36.084504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:36.084504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:36.085597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:36.085597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:36.085597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:36.085597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:36.085597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:36.085597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:36.085597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:36.085597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:36.085597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:36.085597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:36.085597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:36.085597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:36.085597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:36.085597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:36.085597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:36.085597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:36.086557 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:36.086557 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:36.086557 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:36.086557 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:36.086557 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:36.086557 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:36.086557 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:36.086557 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:36.086557 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:36.086557 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:36.086557 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:29:36.086557 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:29:36.091566 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 119/228] 10.222.0.97 () {56 vars in 1114 bytes} [Tue Jul 26 14:29:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:36.103330 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799845c-4329-46c0-9da3-04d24368d5f5 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:36.144084 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971327d6-2e5b-4afd-b9e8-f419d9686c5e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:36.182241 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65be09b-0fbd-4f69-a11c-ab0259bb6506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:36.182888 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65be09b-0fbd-4f69-a11c-ab0259bb6506 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:36.183165 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65be09b-0fbd-4f69-a11c-ab0259bb6506 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:36.183457 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65be09b-0fbd-4f69-a11c-ab0259bb6506 None admin] PATH_INFO: `/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:36.185050 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65be09b-0fbd-4f69-a11c-ab0259bb6506 None admin] RBAC: Authorizing `identity:get_project(project_id=5dbf1af61d63414cbf4c33bdc412a9d6)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:36.186285 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65be09b-0fbd-4f69-a11c-ab0259bb6506 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:36.188399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 110/229] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:35 2022] GET /identity/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6 => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:36.193878 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65be09b-0fbd-4f69-a11c-ab0259bb6506 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:36.222895 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:36.238249 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799845c-4329-46c0-9da3-04d24368d5f5 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:36.277591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796af578-b224-4ad8-a415-0528ad88923f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:36.279087 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796af578-b224-4ad8-a415-0528ad88923f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:36.279696 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796af578-b224-4ad8-a415-0528ad88923f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:36.280143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796af578-b224-4ad8-a415-0528ad88923f None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:36.281426 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796af578-b224-4ad8-a415-0528ad88923f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:36.281879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796af578-b224-4ad8-a415-0528ad88923f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:36.283566 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796af578-b224-4ad8-a415-0528ad88923f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:36.292847 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 120/230] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:29:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:36.299541 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796af578-b224-4ad8-a415-0528ad88923f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:36.322304 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65be09b-0fbd-4f69-a11c-ab0259bb6506 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:36.347878 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:36.421004 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796af578-b224-4ad8-a415-0528ad88923f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:36.456184 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54cbef-39d3-4eec-b8d7-a10d18742a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:36.456830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54cbef-39d3-4eec-b8d7-a10d18742a2c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:36.457114 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54cbef-39d3-4eec-b8d7-a10d18742a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:36.457376 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f6f1a-6457-45e4-8d00-9696217fbf37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:36.457805 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54cbef-39d3-4eec-b8d7-a10d18742a2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:36.458037 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f6f1a-6457-45e4-8d00-9696217fbf37 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:36.458243 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b54cbef-39d3-4eec-b8d7-a10d18742a2c None admin] RBAC: Adding query filter params (role.id=e033961a9363456a8510c34414dd839e, scope.project.id=5dbf1af61d63414cbf4c33bdc412a9d6, user.id=03f24986305c4ed2913113729807ebec) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:36.458346 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f6f1a-6457-45e4-8d00-9696217fbf37 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:36.458560 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b54cbef-39d3-4eec-b8d7-a10d18742a2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:36.458662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f6f1a-6457-45e4-8d00-9696217fbf37 None admin] PATH_INFO: `/v3/users/3539a4db36c24bd980bca0a2a943994e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:36.459787 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f6f1a-6457-45e4-8d00-9696217fbf37 None admin] RBAC: Authorizing `identity:get_user(user_id=3539a4db36c24bd980bca0a2a943994e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:36.460045 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b54cbef-39d3-4eec-b8d7-a10d18742a2c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:36.461225 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f6f1a-6457-45e4-8d00-9696217fbf37 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:36.463135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 121/231] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:36 2022] GET /identity/v3/users/3539a4db36c24bd980bca0a2a943994e => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:36.467859 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 111/232] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:29:36 2022] GET /identity/v3/role_assignments?user.id=03f24986305c4ed2913113729807ebec&scope.project.id=5dbf1af61d63414cbf4c33bdc412a9d6&role.id=e033961a9363456a8510c34414dd839e => generated 259 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:36.473627 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f6f1a-6457-45e4-8d00-9696217fbf37 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:36.604642 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f6f1a-6457-45e4-8d00-9696217fbf37 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:36.606626 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:36.645660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0876de0-05c0-4302-a728-f3693779aa20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:36.646397 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0876de0-05c0-4302-a728-f3693779aa20 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:36.646683 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0876de0-05c0-4302-a728-f3693779aa20 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:36.647016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0876de0-05c0-4302-a728-f3693779aa20 None admin] PATH_INFO: `/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:36.648285 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0876de0-05c0-4302-a728-f3693779aa20 None admin] RBAC: Authorizing `identity:get_project(project_id=5dbf1af61d63414cbf4c33bdc412a9d6)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:36.649755 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0876de0-05c0-4302-a728-f3693779aa20 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:36.651916 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 122/233] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:36 2022] GET /identity/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:36.660566 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54cbef-39d3-4eec-b8d7-a10d18742a2c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:36.781687 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:36.802363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54cbef-39d3-4eec-b8d7-a10d18742a2c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:36.849435 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21bc5d9-1c0d-4905-8d78-6c65fe459291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:36.850121 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bc5d9-1c0d-4905-8d78-6c65fe459291 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:36.850406 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bc5d9-1c0d-4905-8d78-6c65fe459291 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:36.850683 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bc5d9-1c0d-4905-8d78-6c65fe459291 None admin] PATH_INFO: `/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6/users/3539a4db36c24bd980bca0a2a943994e/roles/0602cf371b0f41b390e8a4499ae6bb3c` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:36.852981 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21bc5d9-1c0d-4905-8d78-6c65fe459291 None admin] RBAC: Authorizing `identity:create_grant(project_id=5dbf1af61d63414cbf4c33bdc412a9d6, user_id=3539a4db36c24bd980bca0a2a943994e, role_id=0602cf371b0f41b390e8a4499ae6bb3c)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:36.854496 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21bc5d9-1c0d-4905-8d78-6c65fe459291 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:36.976307 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 112/234] 10.222.0.97 () {58 vars in 1439 bytes} [Tue Jul 26 14:29:36 2022] PUT /identity/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6/users/3539a4db36c24bd980bca0a2a943994e/roles/0602cf371b0f41b390e8a4499ae6bb3c => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:29:37.839343 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a537af6-c454-4b46-add9-36f966e1d3e7 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:37.839878 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a537af6-c454-4b46-add9-36f966e1d3e7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:37.839953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a537af6-c454-4b46-add9-36f966e1d3e7 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:37.841592 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 123/235] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:37.846873 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b983c-a791-41ea-aae7-ec7b86488523 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:37.847270 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b983c-a791-41ea-aae7-ec7b86488523 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:37.847497 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b983c-a791-41ea-aae7-ec7b86488523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:37.872452 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-135b983c-a791-41ea-aae7-ec7b86488523 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:37.873663 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-135b983c-a791-41ea-aae7-ec7b86488523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:37.919190 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea806b9-8936-4d55-aea2-aacfa9c0eb73 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:37.919529 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea806b9-8936-4d55-aea2-aacfa9c0eb73 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:37.919775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea806b9-8936-4d55-aea2-aacfa9c0eb73 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:37.921135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 124/236] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:29:37.926088 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958e6c1-5366-4d1c-8703-6264315e24ca None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:37.926368 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958e6c1-5366-4d1c-8703-6264315e24ca None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:37.926648 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958e6c1-5366-4d1c-8703-6264315e24ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:37.957186 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-8958e6c1-5366-4d1c-8703-6264315e24ca None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:37.958385 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8958e6c1-5366-4d1c-8703-6264315e24ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:37.964108 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 113/237] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:37.989080 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b983c-a791-41ea-aae7-ec7b86488523 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:38.087849 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:38.111186 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 125/238] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:38.123012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b983c-a791-41ea-aae7-ec7b86488523 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:38.135771 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8958e6c1-5366-4d1c-8703-6264315e24ca None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:38.166589 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d7cd6a-c01d-4479-b0d5-a03c5b67853b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:38.167246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7cd6a-c01d-4479-b0d5-a03c5b67853b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:38.167556 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7cd6a-c01d-4479-b0d5-a03c5b67853b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:38.167834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7cd6a-c01d-4479-b0d5-a03c5b67853b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:38.169973 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d7cd6a-c01d-4479-b0d5-a03c5b67853b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:38.171441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d7cd6a-c01d-4479-b0d5-a03c5b67853b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:38.174282 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 114/239] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:29:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:38.182094 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d7cd6a-c01d-4479-b0d5-a03c5b67853b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:38.354219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d7cd6a-c01d-4479-b0d5-a03c5b67853b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:38.357647 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:38.374498 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8958e6c1-5366-4d1c-8703-6264315e24ca None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:38.418533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1a91b-87b5-47a0-9691-bcff0d9674dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:38.419721 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1a91b-87b5-47a0-9691-bcff0d9674dc None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:38.420007 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1a91b-87b5-47a0-9691-bcff0d9674dc None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:38.420283 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1a91b-87b5-47a0-9691-bcff0d9674dc None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:38.420988 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1a91b-87b5-47a0-9691-bcff0d9674dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:38.422425 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1a91b-87b5-47a0-9691-bcff0d9674dc None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:38.446832 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0c78c-d654-49d2-a570-cf58f429f4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:38.447955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0c78c-d654-49d2-a570-cf58f429f4b9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:38.448491 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0c78c-d654-49d2-a570-cf58f429f4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:38.448986 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0c78c-d654-49d2-a570-cf58f429f4b9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:38.460768 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c0c78c-d654-49d2-a570-cf58f429f4b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:38.466160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-f1c0c78c-d654-49d2-a570-cf58f429f4b9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:29:38.466160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:38.466160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:38.466160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:38.466160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:38.466160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:38.466160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:38.466160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:38.466160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:38.466160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:38.466160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:38.466160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:38.466160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:38.466160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:38.466160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:38.466160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:38.467005 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:38.467005 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:38.467005 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:38.467005 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:38.467005 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:38.467005 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:38.467005 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:38.467005 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:38.467005 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:38.467005 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:38.467005 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:38.467005 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:38.467005 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:38.467005 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:38.467005 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:38.467005 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:38.467799 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:38.467799 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:38.467799 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:38.467799 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:38.467799 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:38.467799 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:38.467799 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:38.467799 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:38.467799 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:38.467799 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:38.467799 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:29:38.467799 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:29:38.470431 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 126/240] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:29:38.476463 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0c78c-d654-49d2-a570-cf58f429f4b9 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:38.497787 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:38.510286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 115/241] 10.222.0.97 () {60 vars in 1132 bytes} [Tue Jul 26 14:29:38 2022] POST /identity/v3/users => generated 293 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:29:38.540719 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74cc864-e017-42be-b02a-7d4f62b87b4e None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:38.541113 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74cc864-e017-42be-b02a-7d4f62b87b4e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:38.541470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74cc864-e017-42be-b02a-7d4f62b87b4e None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:38.543239 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 116/242] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:38.549427 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59383613-e05d-4e40-bcde-dd3f1f5e5c70 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:38.549842 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59383613-e05d-4e40-bcde-dd3f1f5e5c70 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:38.550228 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59383613-e05d-4e40-bcde-dd3f1f5e5c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:38.569609 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-59383613-e05d-4e40-bcde-dd3f1f5e5c70 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:38.570985 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-59383613-e05d-4e40-bcde-dd3f1f5e5c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:38.638527 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0c78c-d654-49d2-a570-cf58f429f4b9 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:38.648528 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 117/243] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:29:38.672203 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59383613-e05d-4e40-bcde-dd3f1f5e5c70 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:38.712802 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d21b6-83de-4798-acbe-dea2d21f3db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:38.713454 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d21b6-83de-4798-acbe-dea2d21f3db2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:38.713740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d21b6-83de-4798-acbe-dea2d21f3db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:38.714018 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d21b6-83de-4798-acbe-dea2d21f3db2 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:38.714754 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36d21b6-83de-4798-acbe-dea2d21f3db2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:38.715061 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36d21b6-83de-4798-acbe-dea2d21f3db2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:38.717061 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36d21b6-83de-4798-acbe-dea2d21f3db2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:38.735204 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 127/244] 10.222.0.97 () {56 vars in 1101 bytes} [Tue Jul 26 14:29:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:38.741055 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d21b6-83de-4798-acbe-dea2d21f3db2 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:38.782121 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59383613-e05d-4e40-bcde-dd3f1f5e5c70 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:38.797455 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:38.812519 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:38.821392 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb10f958-f53f-4806-a9f8-fe9da1b5d8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:38.822043 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10f958-f53f-4806-a9f8-fe9da1b5d8f2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:38.822329 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10f958-f53f-4806-a9f8-fe9da1b5d8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:38.822597 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10f958-f53f-4806-a9f8-fe9da1b5d8f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:38.836688 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb10f958-f53f-4806-a9f8-fe9da1b5d8f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:38.842885 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-bb10f958-f53f-4806-a9f8-fe9da1b5d8f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:29:38.842885 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:38.842885 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:38.842885 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:38.842885 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:38.842885 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:38.842885 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:38.842885 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:38.842885 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:38.842885 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:38.842885 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:38.842885 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:38.842885 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:38.842885 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:38.842885 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:38.842885 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:38.847579 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:38.847579 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:38.847579 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:38.847579 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:38.847579 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:38.847579 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:38.847579 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:38.847579 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:38.847579 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:38.847579 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:38.847579 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:38.847579 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:38.847579 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:38.847579 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:38.847579 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:38.847579 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:38.848400 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:38.848400 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:38.848400 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:38.848400 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:38.848400 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:38.848400 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:38.848400 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:38.848400 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:38.848400 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:38.848400 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:38.848400 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:29:38.848400 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:29:38.848400 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 118/245] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:29:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:29:38.851649 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb10f958-f53f-4806-a9f8-fe9da1b5d8f2 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:38.906299 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d21b6-83de-4798-acbe-dea2d21f3db2 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:38.951289 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35f697-b5c2-4766-82b3-2e57630146f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:38.952410 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35f697-b5c2-4766-82b3-2e57630146f1 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:38.952889 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35f697-b5c2-4766-82b3-2e57630146f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:38.953357 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35f697-b5c2-4766-82b3-2e57630146f1 None admin] PATH_INFO: `/v3/users/822f3e0568314a2b8663a80c73fc8830` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:38.971054 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b35f697-b5c2-4766-82b3-2e57630146f1 None admin] RBAC: Authorizing `identity:get_user(user_id=822f3e0568314a2b8663a80c73fc8830)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:38.973020 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b35f697-b5c2-4766-82b3-2e57630146f1 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:38.974954 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 128/246] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:38 2022] GET /identity/v3/users/822f3e0568314a2b8663a80c73fc8830 => generated 297 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:38.976566 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb10f958-f53f-4806-a9f8-fe9da1b5d8f2 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:38.980562 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35f697-b5c2-4766-82b3-2e57630146f1 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:39.020004 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26906838-1590-43d3-9925-f9d3bd15f36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:39.021388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26906838-1590-43d3-9925-f9d3bd15f36b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:39.021700 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26906838-1590-43d3-9925-f9d3bd15f36b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:39.021981 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26906838-1590-43d3-9925-f9d3bd15f36b None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:39.022713 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26906838-1590-43d3-9925-f9d3bd15f36b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:39.023021 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26906838-1590-43d3-9925-f9d3bd15f36b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:39.024428 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26906838-1590-43d3-9925-f9d3bd15f36b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:39.035061 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 119/247] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:39.041407 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26906838-1590-43d3-9925-f9d3bd15f36b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:39.084793 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:39.114299 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35f697-b5c2-4766-82b3-2e57630146f1 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:39.121327 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:39.159621 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d8462c-fa6b-42f9-9108-17859f8fdebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:39.160457 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8462c-fa6b-42f9-9108-17859f8fdebb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:39.160759 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8462c-fa6b-42f9-9108-17859f8fdebb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:39.161034 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8462c-fa6b-42f9-9108-17859f8fdebb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:39.171731 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26906838-1590-43d3-9925-f9d3bd15f36b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:39.226565 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11b8b2-2a2e-4708-97d4-20b8352529ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:39.227697 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11b8b2-2a2e-4708-97d4-20b8352529ee None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:39.228179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11b8b2-2a2e-4708-97d4-20b8352529ee None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:39.228674 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11b8b2-2a2e-4708-97d4-20b8352529ee None admin] PATH_INFO: `/v3/users/3539a4db36c24bd980bca0a2a943994e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:39.230086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11b8b2-2a2e-4708-97d4-20b8352529ee None admin] RBAC: Authorizing `identity:get_user(user_id=3539a4db36c24bd980bca0a2a943994e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:39.232393 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11b8b2-2a2e-4708-97d4-20b8352529ee None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:39.234534 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 120/248] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:39 2022] GET /identity/v3/users/3539a4db36c24bd980bca0a2a943994e => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:39.236125 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840356f-3f46-40c9-ac5f-cd520badf11a None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:39.236613 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840356f-3f46-40c9-ac5f-cd520badf11a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:39.237081 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840356f-3f46-40c9-ac5f-cd520badf11a None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:39.238815 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 121/249] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:39.244809 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cacc74-8e14-43a0-9577-209a9153a71b None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:39.245289 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cacc74-8e14-43a0-9577-209a9153a71b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:39.245733 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cacc74-8e14-43a0-9577-209a9153a71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:39.257702 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d8462c-fa6b-42f9-9108-17859f8fdebb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:39.259282 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d8462c-fa6b-42f9-9108-17859f8fdebb None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:39.266015 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-95cacc74-8e14-43a0-9577-209a9153a71b None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:39.267472 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-95cacc74-8e14-43a0-9577-209a9153a71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:39.341402 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 122/250] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:29:39.342670 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cacc74-8e14-43a0-9577-209a9153a71b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:39.350467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:39.391658 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 129/251] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:29:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:29:39.392788 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d8462c-fa6b-42f9-9108-17859f8fdebb None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:39.398364 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:39.452318 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cacc74-8e14-43a0-9577-209a9153a71b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:39.487381 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8aeaa8-4a3b-408a-9143-ba9bc3de1696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:39.488033 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8aeaa8-4a3b-408a-9143-ba9bc3de1696 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:39.488321 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8aeaa8-4a3b-408a-9143-ba9bc3de1696 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:39.488596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8aeaa8-4a3b-408a-9143-ba9bc3de1696 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:39.526101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d8462c-fa6b-42f9-9108-17859f8fdebb None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:39.560361 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8aeaa8-4a3b-408a-9143-ba9bc3de1696 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:39.561541 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8aeaa8-4a3b-408a-9143-ba9bc3de1696 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:39.563703 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b08f35-c1d5-40f4-acfd-99a3a97d55af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:39.564633 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b08f35-c1d5-40f4-acfd-99a3a97d55af None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:39.565041 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b08f35-c1d5-40f4-acfd-99a3a97d55af None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:39.565459 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b08f35-c1d5-40f4-acfd-99a3a97d55af None admin] PATH_INFO: `/v3/roles/member` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:39.577008 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b08f35-c1d5-40f4-acfd-99a3a97d55af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:39.582286 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-d5b08f35-c1d5-40f4-acfd-99a3a97d55af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:39.582286 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:39.582286 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:39.582286 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:39.582286 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:39.582286 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:39.582286 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:39.582286 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:39.582286 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:39.582286 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:39.582286 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:39.582286 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:39.582286 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:39.582286 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:39.582286 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:39.582286 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:39.583326 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:39.583326 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:39.583326 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:39.583326 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:39.583326 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:39.583326 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:39.583326 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:39.583326 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:39.583326 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:39.583326 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:39.583326 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:39.583326 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:39.583326 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:39.583326 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:39.583326 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:39.583326 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:39.584182 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:39.584182 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:39.584182 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:39.584182 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:39.584182 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:39.584182 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:39.584182 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:39.584182 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:39.584182 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:39.584182 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:39.584182 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:39.584182 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:29:39.586225 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 130/252] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:39.592674 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b08f35-c1d5-40f4-acfd-99a3a97d55af None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:39.612100 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:39.639060 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:39.674677 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 123/253] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:29:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:29:39.674677 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ad4b8-0d62-4bbe-8820-1c80c102be25 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:39.674677 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ad4b8-0d62-4bbe-8820-1c80c102be25 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:39.674677 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ad4b8-0d62-4bbe-8820-1c80c102be25 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:39.674677 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 124/254] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:39.674677 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ad4b8-0d62-4bbe-8820-1c80c102be25 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:39.708519 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b08f35-c1d5-40f4-acfd-99a3a97d55af None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:39.747541 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42412d-4e18-4ebc-8b8f-160e4ae2014c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:39.748511 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42412d-4e18-4ebc-8b8f-160e4ae2014c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:39.748977 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42412d-4e18-4ebc-8b8f-160e4ae2014c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:39.749419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42412d-4e18-4ebc-8b8f-160e4ae2014c None admin] PATH_INFO: `/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:39.750745 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be42412d-4e18-4ebc-8b8f-160e4ae2014c None admin] RBAC: Authorizing `identity:get_project(project_id=5dbf1af61d63414cbf4c33bdc412a9d6)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:39.752400 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be42412d-4e18-4ebc-8b8f-160e4ae2014c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:39.754645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 131/255] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:39 2022] GET /identity/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:39.755624 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42412d-4e18-4ebc-8b8f-160e4ae2014c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:39.806870 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ad4b8-0d62-4bbe-8820-1c80c102be25 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:39.851315 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03aaac-2289-4a57-ae87-6979611a1979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:39.851964 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03aaac-2289-4a57-ae87-6979611a1979 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:39.852253 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03aaac-2289-4a57-ae87-6979611a1979 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:39.852533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03aaac-2289-4a57-ae87-6979611a1979 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:39.853595 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e03aaac-2289-4a57-ae87-6979611a1979 None admin] RBAC: Adding query filter params (name=member) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:39.854049 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e03aaac-2289-4a57-ae87-6979611a1979 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:39.855707 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e03aaac-2289-4a57-ae87-6979611a1979 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:39.862597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42412d-4e18-4ebc-8b8f-160e4ae2014c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:39.867432 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 125/256] 10.222.0.97 () {56 vars in 1101 bytes} [Tue Jul 26 14:29:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:39.872290 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26f746-023d-4907-8ccd-c7e9e16fe3fa None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:39.872290 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26f746-023d-4907-8ccd-c7e9e16fe3fa None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:39.872290 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26f746-023d-4907-8ccd-c7e9e16fe3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:39.878937 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:39.890933 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-9d26f746-023d-4907-8ccd-c7e9e16fe3fa None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:39.892618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9d26f746-023d-4907-8ccd-c7e9e16fe3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:39.903310 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:39.907263 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa881506-bb37-4d6b-8a16-476639bb539d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:39.908469 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa881506-bb37-4d6b-8a16-476639bb539d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:39.908948 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa881506-bb37-4d6b-8a16-476639bb539d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:39.909389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa881506-bb37-4d6b-8a16-476639bb539d None admin] PATH_INFO: `/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:39.910793 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa881506-bb37-4d6b-8a16-476639bb539d None admin] RBAC: Authorizing `identity:get_project(project_id=5dbf1af61d63414cbf4c33bdc412a9d6)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:39.912523 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa881506-bb37-4d6b-8a16-476639bb539d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:39.914439 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 132/257] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:39 2022] GET /identity/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:39.915427 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa881506-bb37-4d6b-8a16-476639bb539d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:39.974752 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 126/258] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:39.975547 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26f746-023d-4907-8ccd-c7e9e16fe3fa None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:40.038519 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa881506-bb37-4d6b-8a16-476639bb539d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:40.081697 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec12c7-ee75-490d-a7b3-48726b1c0154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:40.083084 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec12c7-ee75-490d-a7b3-48726b1c0154 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:40.083377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec12c7-ee75-490d-a7b3-48726b1c0154 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:40.083700 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec12c7-ee75-490d-a7b3-48726b1c0154 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:40.084492 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec12c7-ee75-490d-a7b3-48726b1c0154 None admin] RBAC: Adding query filter params (role.id=2b922104c92a4cbc9432fbce1c9bda1e, scope.project.id=5dbf1af61d63414cbf4c33bdc412a9d6, user.id=822f3e0568314a2b8663a80c73fc8830) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:40.084805 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec12c7-ee75-490d-a7b3-48726b1c0154 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:40.086332 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec12c7-ee75-490d-a7b3-48726b1c0154 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:40.093788 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 133/259] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:29:39 2022] GET /identity/v3/role_assignments?user.id=822f3e0568314a2b8663a80c73fc8830&scope.project.id=5dbf1af61d63414cbf4c33bdc412a9d6&role.id=2b922104c92a4cbc9432fbce1c9bda1e => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:40.094394 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec12c7-ee75-490d-a7b3-48726b1c0154 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:40.131672 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26f746-023d-4907-8ccd-c7e9e16fe3fa None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:40.178056 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:40.192959 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc002e47-5467-4bee-b294-683091084ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:40.195812 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc002e47-5467-4bee-b294-683091084ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:40.197633 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc002e47-5467-4bee-b294-683091084ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:40.198158 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc002e47-5467-4bee-b294-683091084ce5 None admin] PATH_INFO: `/v3/users/03f24986305c4ed2913113729807ebec` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:40.201215 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc002e47-5467-4bee-b294-683091084ce5 None admin] RBAC: Authorizing `identity:get_user(user_id=03f24986305c4ed2913113729807ebec)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:40.204632 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc002e47-5467-4bee-b294-683091084ce5 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:40.206743 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 127/260] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:39 2022] GET /identity/v3/users/03f24986305c4ed2913113729807ebec => generated 297 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:40.211878 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc002e47-5467-4bee-b294-683091084ce5 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:40.239297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:40.315274 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec12c7-ee75-490d-a7b3-48726b1c0154 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:40.385425 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3696e68e-0c48-43b3-80aa-a027c10fb861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:40.386606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696e68e-0c48-43b3-80aa-a027c10fb861 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:40.387925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696e68e-0c48-43b3-80aa-a027c10fb861 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:40.388936 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696e68e-0c48-43b3-80aa-a027c10fb861 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:40.390078 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3696e68e-0c48-43b3-80aa-a027c10fb861 None admin] RBAC: Adding query filter params (role.id=51b2d10dfffb4b75b186f3d8b15e2a78, scope.project.id=5dbf1af61d63414cbf4c33bdc412a9d6, user.id=3539a4db36c24bd980bca0a2a943994e) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:40.392115 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc002e47-5467-4bee-b294-683091084ce5 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:40.392640 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3696e68e-0c48-43b3-80aa-a027c10fb861 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:40.394542 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3696e68e-0c48-43b3-80aa-a027c10fb861 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:40.409184 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 134/261] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:29:40 2022] GET /identity/v3/role_assignments?user.id=3539a4db36c24bd980bca0a2a943994e&scope.project.id=5dbf1af61d63414cbf4c33bdc412a9d6&role.id=51b2d10dfffb4b75b186f3d8b15e2a78 => generated 608 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:40.411957 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3696e68e-0c48-43b3-80aa-a027c10fb861 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:40.464892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef139b-bf69-4828-aa4b-998be8af5673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:40.465897 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef139b-bf69-4828-aa4b-998be8af5673 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:40.466184 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef139b-bf69-4828-aa4b-998be8af5673 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:40.466463 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef139b-bf69-4828-aa4b-998be8af5673 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:40.488760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef139b-bf69-4828-aa4b-998be8af5673 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:40.493969 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-a5ef139b-bf69-4828-aa4b-998be8af5673 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:29:40.493969 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:40.493969 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:40.493969 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:40.493969 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:40.493969 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:40.493969 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:40.493969 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:40.493969 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:40.493969 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:40.493969 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:40.493969 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:40.493969 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:40.493969 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:40.493969 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:40.493969 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:40.494811 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:40.494811 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:40.494811 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:40.494811 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:40.494811 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:40.494811 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:40.494811 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:40.494811 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:40.494811 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:40.494811 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:40.494811 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:40.494811 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:40.494811 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:40.494811 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:40.494811 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:40.494811 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:40.499909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:40.499909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:40.499909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:40.499909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:40.499909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:40.499909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:40.499909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:40.499909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:40.499909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:40.499909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:40.499909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:29:40.499909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:29:40.499909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 128/262] 10.222.0.97 () {56 vars in 1114 bytes} [Tue Jul 26 14:29:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:40.518754 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef139b-bf69-4828-aa4b-998be8af5673 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:40.556809 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3696e68e-0c48-43b3-80aa-a027c10fb861 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:40.559788 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:40.595939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125be36a-243e-45b6-8c65-e6707056224c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:40.605087 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125be36a-243e-45b6-8c65-e6707056224c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:40.605792 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125be36a-243e-45b6-8c65-e6707056224c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:40.606789 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125be36a-243e-45b6-8c65-e6707056224c None admin] PATH_INFO: `/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:40.608386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125be36a-243e-45b6-8c65-e6707056224c None admin] RBAC: Authorizing `identity:get_project(project_id=5dbf1af61d63414cbf4c33bdc412a9d6)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:40.610273 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125be36a-243e-45b6-8c65-e6707056224c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:40.612368 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 135/263] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:40 2022] GET /identity/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:40.618266 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125be36a-243e-45b6-8c65-e6707056224c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:40.624075 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:40.658028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef139b-bf69-4828-aa4b-998be8af5673 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:40.694941 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67750ee5-0a2e-4fce-93ab-babcf598b05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:40.695613 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67750ee5-0a2e-4fce-93ab-babcf598b05c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:40.695902 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67750ee5-0a2e-4fce-93ab-babcf598b05c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:40.696178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67750ee5-0a2e-4fce-93ab-babcf598b05c None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:40.696910 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67750ee5-0a2e-4fce-93ab-babcf598b05c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:40.697217 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67750ee5-0a2e-4fce-93ab-babcf598b05c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:40.698641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67750ee5-0a2e-4fce-93ab-babcf598b05c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:40.708706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 129/264] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:29:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:40.714905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67750ee5-0a2e-4fce-93ab-babcf598b05c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:40.828335 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67750ee5-0a2e-4fce-93ab-babcf598b05c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:40.864098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:40.869807 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a8c8c-697a-4135-af0a-cc6edeb8e9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:40.870468 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a8c8c-697a-4135-af0a-cc6edeb8e9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:40.870762 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a8c8c-697a-4135-af0a-cc6edeb8e9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:40.871094 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a8c8c-697a-4135-af0a-cc6edeb8e9a9 None admin] PATH_INFO: `/v3/users/3539a4db36c24bd980bca0a2a943994e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:40.873066 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a8c8c-697a-4135-af0a-cc6edeb8e9a9 None admin] RBAC: Authorizing `identity:get_user(user_id=3539a4db36c24bd980bca0a2a943994e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:40.874552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a8c8c-697a-4135-af0a-cc6edeb8e9a9 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:40.876431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 130/265] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:40 2022] GET /identity/v3/users/3539a4db36c24bd980bca0a2a943994e => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:40.883073 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a8c8c-697a-4135-af0a-cc6edeb8e9a9 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:40.918537 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125be36a-243e-45b6-8c65-e6707056224c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:40.979879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ccf2d-0345-4366-b661-86bd027a619c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:40.980573 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ccf2d-0345-4366-b661-86bd027a619c None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:40.980857 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ccf2d-0345-4366-b661-86bd027a619c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:40.981208 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ccf2d-0345-4366-b661-86bd027a619c None admin] PATH_INFO: `/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6/users/03f24986305c4ed2913113729807ebec/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:40.985162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95ccf2d-0345-4366-b661-86bd027a619c None admin] RBAC: Authorizing `identity:create_grant(project_id=5dbf1af61d63414cbf4c33bdc412a9d6, user_id=03f24986305c4ed2913113729807ebec, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:40.986626 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95ccf2d-0345-4366-b661-86bd027a619c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:41.037685 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a8c8c-697a-4135-af0a-cc6edeb8e9a9 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:41.070611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 136/266] 10.222.0.97 () {58 vars in 1439 bytes} [Tue Jul 26 14:29:40 2022] PUT /identity/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6/users/03f24986305c4ed2913113729807ebec/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:29:41.076788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df3b24b-239f-4c25-ac96-ec4a4a595236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:41.077479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3b24b-239f-4c25-ac96-ec4a4a595236 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:41.077764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3b24b-239f-4c25-ac96-ec4a4a595236 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:41.078036 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3b24b-239f-4c25-ac96-ec4a4a595236 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:41.156694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:41.159875 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df3b24b-239f-4c25-ac96-ec4a4a595236 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:41.161368 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df3b24b-239f-4c25-ac96-ec4a4a595236 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:41.275486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 131/267] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:29:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:29:41.277338 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ccf2d-0345-4366-b661-86bd027a619c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:41.331247 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:41.422343 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ccf2d-0345-4366-b661-86bd027a619c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:41.452919 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d882b11-7ce6-4c97-b37f-36f0ba80cc9e None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:41.453217 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d882b11-7ce6-4c97-b37f-36f0ba80cc9e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:41.453469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d882b11-7ce6-4c97-b37f-36f0ba80cc9e None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:41.454833 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 132/268] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:29:41.459865 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd60b3e-9e72-4059-9e68-d9c8ff444d51 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:41.460152 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd60b3e-9e72-4059-9e68-d9c8ff444d51 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:41.460428 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd60b3e-9e72-4059-9e68-d9c8ff444d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:41.471172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce2855-430b-4c0e-bfbf-915f59eac085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:41.471851 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce2855-430b-4c0e-bfbf-915f59eac085 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:41.472131 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce2855-430b-4c0e-bfbf-915f59eac085 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:41.472404 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce2855-430b-4c0e-bfbf-915f59eac085 None admin] PATH_INFO: `/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:41.473971 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ce2855-430b-4c0e-bfbf-915f59eac085 None admin] RBAC: Authorizing `identity:get_project(project_id=5dbf1af61d63414cbf4c33bdc412a9d6)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:41.475761 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ce2855-430b-4c0e-bfbf-915f59eac085 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:41.477595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 137/269] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:41 2022] GET /identity/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:41.479307 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-efd60b3e-9e72-4059-9e68-d9c8ff444d51 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:41.479953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a5abc-0695-4e0f-ae88-7f9eceefed22 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:41.480248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a5abc-0695-4e0f-ae88-7f9eceefed22 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:41.480527 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a5abc-0695-4e0f-ae88-7f9eceefed22 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:41.480624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-efd60b3e-9e72-4059-9e68-d9c8ff444d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:41.482592 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 138/270] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:29:41.483521 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a5abc-0695-4e0f-ae88-7f9eceefed22 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:41.575550 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 133/271] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:29:41.577230 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa841c41-5727-4f86-9b3e-292fab905596 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:41.577721 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa841c41-5727-4f86-9b3e-292fab905596 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:41.578219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa841c41-5727-4f86-9b3e-292fab905596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:41.607778 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:41.608926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-fa841c41-5727-4f86-9b3e-292fab905596 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:41.610333 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fa841c41-5727-4f86-9b3e-292fab905596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:41.612193 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a5abc-0695-4e0f-ae88-7f9eceefed22 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:41.636741 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:41.658284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1244b4a7-50a9-4f3d-b86b-ab6a5405c4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:41.658972 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244b4a7-50a9-4f3d-b86b-ab6a5405c4eb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:41.659730 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244b4a7-50a9-4f3d-b86b-ab6a5405c4eb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:41.660034 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244b4a7-50a9-4f3d-b86b-ab6a5405c4eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:41.660831 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1244b4a7-50a9-4f3d-b86b-ab6a5405c4eb None admin] RBAC: Adding query filter params (role.id=0602cf371b0f41b390e8a4499ae6bb3c, scope.project.id=5dbf1af61d63414cbf4c33bdc412a9d6, user.id=3539a4db36c24bd980bca0a2a943994e) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:41.661140 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1244b4a7-50a9-4f3d-b86b-ab6a5405c4eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:41.665747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1244b4a7-50a9-4f3d-b86b-ab6a5405c4eb None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:41.671199 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 139/272] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:29:41 2022] GET /identity/v3/role_assignments?user.id=3539a4db36c24bd980bca0a2a943994e&scope.project.id=5dbf1af61d63414cbf4c33bdc412a9d6&role.id=0602cf371b0f41b390e8a4499ae6bb3c => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:41.672609 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1244b4a7-50a9-4f3d-b86b-ab6a5405c4eb None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:41.703295 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 134/273] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:41.744881 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa841c41-5727-4f86-9b3e-292fab905596 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:41.818081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1244b4a7-50a9-4f3d-b86b-ab6a5405c4eb None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:41.880155 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa841c41-5727-4f86-9b3e-292fab905596 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:41.884941 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6560b3a4-f3f8-4694-9c13-c1d91fbaf5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:41.885789 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6560b3a4-f3f8-4694-9c13-c1d91fbaf5fb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:41.886218 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6560b3a4-f3f8-4694-9c13-c1d91fbaf5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:41.886663 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6560b3a4-f3f8-4694-9c13-c1d91fbaf5fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:41.898248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6560b3a4-f3f8-4694-9c13-c1d91fbaf5fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:41.904308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-6560b3a4-f3f8-4694-9c13-c1d91fbaf5fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:41.904308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:41.904308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:41.904308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:41.904308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:41.904308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:41.904308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:41.904308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:41.904308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:41.904308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:41.904308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:41.904308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:41.904308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:41.904308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:41.904308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:41.904308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:41.905172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:41.905172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:41.905172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:41.905172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:41.905172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:41.905172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:41.905172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:41.905172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:41.905172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:41.905172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:41.905172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:41.905172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:41.905172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:41.905172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:41.905172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:41.905172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:41.905943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:41.905943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:41.905943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:41.905943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:41.905943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:41.905943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:41.905943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:41.905943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:41.905943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:41.905943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:41.905943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:41.905943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:29:41.908647 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 140/274] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:41.914414 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6560b3a4-f3f8-4694-9c13-c1d91fbaf5fb None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:41.921675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64286f-85b1-44f0-ba1e-8e85ee848e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:41.922402 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64286f-85b1-44f0-ba1e-8e85ee848e06 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:41.922808 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64286f-85b1-44f0-ba1e-8e85ee848e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:41.923091 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64286f-85b1-44f0-ba1e-8e85ee848e06 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:41.924336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d64286f-85b1-44f0-ba1e-8e85ee848e06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:41.925876 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d64286f-85b1-44f0-ba1e-8e85ee848e06 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:41.927704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 135/275] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:29:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:41.934555 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64286f-85b1-44f0-ba1e-8e85ee848e06 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:41.947611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:41.966797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:42.034874 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6560b3a4-f3f8-4694-9c13-c1d91fbaf5fb None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:42.041562 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64286f-85b1-44f0-ba1e-8e85ee848e06 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:42.080031 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e1f11-4725-4267-8b6f-96dcf0fc05c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:42.080842 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e1f11-4725-4267-8b6f-96dcf0fc05c6 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:42.081202 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e1f11-4725-4267-8b6f-96dcf0fc05c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:42.081608 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e1f11-4725-4267-8b6f-96dcf0fc05c6 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:42.082490 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875e1f11-4725-4267-8b6f-96dcf0fc05c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:42.084153 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875e1f11-4725-4267-8b6f-96dcf0fc05c6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:42.084262 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880dfd8-c99b-44f1-a8c1-ff4aa17c0156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:42.085139 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880dfd8-c99b-44f1-a8c1-ff4aa17c0156 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:42.085609 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880dfd8-c99b-44f1-a8c1-ff4aa17c0156 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:42.086057 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880dfd8-c99b-44f1-a8c1-ff4aa17c0156 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:42.086962 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c880dfd8-c99b-44f1-a8c1-ff4aa17c0156 None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:42.087473 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c880dfd8-c99b-44f1-a8c1-ff4aa17c0156 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:42.089099 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c880dfd8-c99b-44f1-a8c1-ff4aa17c0156 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:42.099053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 141/276] 10.222.0.97 () {56 vars in 1101 bytes} [Tue Jul 26 14:29:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:42.104830 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880dfd8-c99b-44f1-a8c1-ff4aa17c0156 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:42.195839 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:42.214969 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880dfd8-c99b-44f1-a8c1-ff4aa17c0156 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:42.250916 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd3da5f-ace6-40c1-ac13-6220a3ae62ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:42.251795 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3da5f-ace6-40c1-ac13-6220a3ae62ee None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:42.252084 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3da5f-ace6-40c1-ac13-6220a3ae62ee None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:42.252371 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3da5f-ace6-40c1-ac13-6220a3ae62ee None admin] PATH_INFO: `/v3/users/07a7f8d10eb44197a4335c6f76dc9c9b` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:42.257318 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 136/277] 10.222.0.97 () {60 vars in 1132 bytes} [Tue Jul 26 14:29:41 2022] POST /identity/v3/users => generated 293 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:29:42.270501 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd3da5f-ace6-40c1-ac13-6220a3ae62ee None admin] RBAC: Authorizing `identity:get_user(user_id=07a7f8d10eb44197a4335c6f76dc9c9b)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:42.272053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd3da5f-ace6-40c1-ac13-6220a3ae62ee None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:42.274077 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 142/278] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:42 2022] GET /identity/v3/users/07a7f8d10eb44197a4335c6f76dc9c9b => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:42.280054 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e1f11-4725-4267-8b6f-96dcf0fc05c6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:42.378553 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:42.396018 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e1f11-4725-4267-8b6f-96dcf0fc05c6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:42.434990 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb73d16-51cb-4f28-8dd1-6e5fb6b4d7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:42.435724 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb73d16-51cb-4f28-8dd1-6e5fb6b4d7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:42.436004 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb73d16-51cb-4f28-8dd1-6e5fb6b4d7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:42.436318 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb73d16-51cb-4f28-8dd1-6e5fb6b4d7e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:42.437114 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb73d16-51cb-4f28-8dd1-6e5fb6b4d7e3 None admin] RBAC: Adding query filter params (role.id=e033961a9363456a8510c34414dd839e, user.id=07a7f8d10eb44197a4335c6f76dc9c9b, scope.system=all) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:42.437425 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb73d16-51cb-4f28-8dd1-6e5fb6b4d7e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:42.438853 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb73d16-51cb-4f28-8dd1-6e5fb6b4d7e3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:42.444703 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 137/279] 10.222.0.97 () {56 vars in 1308 bytes} [Tue Jul 26 14:29:42 2022] GET /identity/v3/role_assignments?user.id=07a7f8d10eb44197a4335c6f76dc9c9b&scope.system=all&role.id=e033961a9363456a8510c34414dd839e => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:42.579557 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc6e58-f522-4380-acf7-df99380dcb38 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:42.579932 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc6e58-f522-4380-acf7-df99380dcb38 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:42.580169 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc6e58-f522-4380-acf7-df99380dcb38 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:42.581866 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 143/280] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:42.587369 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325f7f2-d043-4e71-b566-fed57156b816 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:42.587683 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325f7f2-d043-4e71-b566-fed57156b816 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:42.587948 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325f7f2-d043-4e71-b566-fed57156b816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:42.609786 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-9325f7f2-d043-4e71-b566-fed57156b816 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:42.610968 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9325f7f2-d043-4e71-b566-fed57156b816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:42.692766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 138/281] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:29:42.728019 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc6e58-f522-4380-acf7-df99380dcb38 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:42.863642 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc6e58-f522-4380-acf7-df99380dcb38 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:42.898938 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91185caf-5226-412d-9ed9-0e128c44363f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:42.899787 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91185caf-5226-412d-9ed9-0e128c44363f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:42.900066 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91185caf-5226-412d-9ed9-0e128c44363f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:42.900340 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91185caf-5226-412d-9ed9-0e128c44363f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:42.911172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91185caf-5226-412d-9ed9-0e128c44363f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:42.916200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-91185caf-5226-412d-9ed9-0e128c44363f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:29:42.916200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:42.916200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:42.916200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:42.916200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:42.916200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:42.916200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:42.916200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:42.916200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:42.916200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:42.916200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:42.916200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:42.916200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:42.916200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:42.916200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:42.916200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:42.917073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:42.917073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:42.917073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:42.917073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:42.917073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:42.917073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:42.917073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:42.917073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:42.917073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:42.917073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:42.917073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:42.917073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:42.917073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:42.917073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:42.917073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:42.917073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:42.917851 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:42.917851 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:42.917851 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:42.917851 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:42.917851 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:42.917851 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:42.917851 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:42.917851 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:42.917851 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:42.917851 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:42.917851 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:29:42.917851 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:29:42.917851 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 144/282] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:42.923761 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325f7f2-d043-4e71-b566-fed57156b816 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:43.048231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:43.065102 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325f7f2-d043-4e71-b566-fed57156b816 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:43.112535 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4374ef8-6238-411d-b59e-20749b3e83a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:43.113163 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4374ef8-6238-411d-b59e-20749b3e83a9 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:43.113447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4374ef8-6238-411d-b59e-20749b3e83a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:43.113777 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4374ef8-6238-411d-b59e-20749b3e83a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:43.114437 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4374ef8-6238-411d-b59e-20749b3e83a9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:43.114741 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4374ef8-6238-411d-b59e-20749b3e83a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:43.116143 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4374ef8-6238-411d-b59e-20749b3e83a9 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:43.129970 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 139/283] 10.222.0.97 () {56 vars in 1101 bytes} [Tue Jul 26 14:29:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:43.135649 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91185caf-5226-412d-9ed9-0e128c44363f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:43.147746 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:43.306278 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91185caf-5226-412d-9ed9-0e128c44363f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:43.351238 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d620e-25c4-4ccf-bcdc-bdd967a9c878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:43.351905 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d620e-25c4-4ccf-bcdc-bdd967a9c878 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:43.352183 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d620e-25c4-4ccf-bcdc-bdd967a9c878 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:43.352455 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d620e-25c4-4ccf-bcdc-bdd967a9c878 None admin] PATH_INFO: `/v3/users/822f3e0568314a2b8663a80c73fc8830` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:43.356207 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d620e-25c4-4ccf-bcdc-bdd967a9c878 None admin] RBAC: Authorizing `identity:get_user(user_id=822f3e0568314a2b8663a80c73fc8830)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:43.357674 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d620e-25c4-4ccf-bcdc-bdd967a9c878 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:43.359659 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 145/284] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:43 2022] GET /identity/v3/users/822f3e0568314a2b8663a80c73fc8830 => generated 297 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:43.364776 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4374ef8-6238-411d-b59e-20749b3e83a9 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:43.435492 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa42975-a5ff-4cd6-9442-0c87623a4fff None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:43.435808 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa42975-a5ff-4cd6-9442-0c87623a4fff None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:43.436057 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa42975-a5ff-4cd6-9442-0c87623a4fff None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:43.437667 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 146/285] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:43.442488 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98caab8-f158-4c3d-b735-292d9bf93c80 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:43.442774 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98caab8-f158-4c3d-b735-292d9bf93c80 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:43.443049 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98caab8-f158-4c3d-b735-292d9bf93c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:43.460911 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-d98caab8-f158-4c3d-b735-292d9bf93c80 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:43.462116 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d98caab8-f158-4c3d-b735-292d9bf93c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:43.533862 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4374ef8-6238-411d-b59e-20749b3e83a9 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:43.569640 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 147/286] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:29:43.594828 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98caab8-f158-4c3d-b735-292d9bf93c80 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:43.598097 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:43.613795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5449f066-5b23-45c7-ae41-3021a4adef27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:43.615884 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449f066-5b23-45c7-ae41-3021a4adef27 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:43.617099 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449f066-5b23-45c7-ae41-3021a4adef27 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:43.617573 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449f066-5b23-45c7-ae41-3021a4adef27 None admin] PATH_INFO: `/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:43.622054 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5449f066-5b23-45c7-ae41-3021a4adef27 None admin] RBAC: Authorizing `identity:get_project(project_id=5dbf1af61d63414cbf4c33bdc412a9d6)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:43.623732 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5449f066-5b23-45c7-ae41-3021a4adef27 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:43.625870 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 140/287] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:43 2022] GET /identity/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6 => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:43.631371 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5449f066-5b23-45c7-ae41-3021a4adef27 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:43.643526 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:43.742846 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5449f066-5b23-45c7-ae41-3021a4adef27 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:43.780389 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1180c129-2b5a-46d7-bf55-ddfa7d556c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:43.781354 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180c129-2b5a-46d7-bf55-ddfa7d556c40 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:43.781817 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180c129-2b5a-46d7-bf55-ddfa7d556c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:43.782562 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180c129-2b5a-46d7-bf55-ddfa7d556c40 None admin] PATH_INFO: `/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6/users/822f3e0568314a2b8663a80c73fc8830/roles/2b922104c92a4cbc9432fbce1c9bda1e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:43.785406 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1180c129-2b5a-46d7-bf55-ddfa7d556c40 None admin] RBAC: Authorizing `identity:create_grant(project_id=5dbf1af61d63414cbf4c33bdc412a9d6, user_id=822f3e0568314a2b8663a80c73fc8830, role_id=2b922104c92a4cbc9432fbce1c9bda1e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:43.787206 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1180c129-2b5a-46d7-bf55-ddfa7d556c40 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:43.826924 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98caab8-f158-4c3d-b735-292d9bf93c80 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:43.860443 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 141/288] 10.222.0.97 () {58 vars in 1439 bytes} [Tue Jul 26 14:29:43 2022] PUT /identity/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6/users/822f3e0568314a2b8663a80c73fc8830/roles/2b922104c92a4cbc9432fbce1c9bda1e => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:29:43.874629 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6591dba-711b-4453-b8b6-b7ba208327ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:43.875290 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6591dba-711b-4453-b8b6-b7ba208327ec None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:43.875660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6591dba-711b-4453-b8b6-b7ba208327ec None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:43.875978 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6591dba-711b-4453-b8b6-b7ba208327ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:43.893938 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6591dba-711b-4453-b8b6-b7ba208327ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:43.899774 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-e6591dba-711b-4453-b8b6-b7ba208327ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:43.899774 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:43.899774 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:43.899774 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:43.899774 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:43.899774 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:43.899774 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:43.899774 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:43.899774 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:43.899774 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:43.899774 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:43.899774 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:43.899774 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:43.899774 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:43.899774 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:43.899774 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:43.906673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:43.906673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:43.906673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:43.906673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:43.906673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:43.906673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:43.906673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:43.906673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:43.906673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:43.906673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:43.906673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:43.906673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:43.906673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:43.906673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:43.906673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:43.906673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:43.907507 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:43.907507 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:43.907507 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:43.907507 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:43.907507 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:43.907507 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:43.907507 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:43.907507 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:43.907507 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:43.907507 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:43.907507 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:43.907507 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:29:43.908092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 148/289] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:43.914566 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1180c129-2b5a-46d7-bf55-ddfa7d556c40 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:44.053293 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:44.097178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1180c129-2b5a-46d7-bf55-ddfa7d556c40 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:44.136150 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dabfb3-c690-44d6-b3dc-1b34cc51ef66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:44.136738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dabfb3-c690-44d6-b3dc-1b34cc51ef66 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:44.137686 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dabfb3-c690-44d6-b3dc-1b34cc51ef66 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:44.137985 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dabfb3-c690-44d6-b3dc-1b34cc51ef66 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:44.138729 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dabfb3-c690-44d6-b3dc-1b34cc51ef66 None admin] RBAC: Adding query filter params (name=member) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:44.139069 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dabfb3-c690-44d6-b3dc-1b34cc51ef66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:44.140676 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dabfb3-c690-44d6-b3dc-1b34cc51ef66 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:44.150945 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 142/290] 10.222.0.97 () {56 vars in 1101 bytes} [Tue Jul 26 14:29:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:44.156933 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6591dba-711b-4453-b8b6-b7ba208327ec None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:44.228408 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:44.274861 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6591dba-711b-4453-b8b6-b7ba208327ec None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:44.309717 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51606bac-0ff2-48fd-82e0-89284fff0373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:44.310379 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51606bac-0ff2-48fd-82e0-89284fff0373 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:44.310656 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51606bac-0ff2-48fd-82e0-89284fff0373 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:44.310931 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51606bac-0ff2-48fd-82e0-89284fff0373 None admin] PATH_INFO: `/v3/users/03f24986305c4ed2913113729807ebec` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:44.312144 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51606bac-0ff2-48fd-82e0-89284fff0373 None admin] RBAC: Authorizing `identity:get_user(user_id=03f24986305c4ed2913113729807ebec)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:44.313608 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51606bac-0ff2-48fd-82e0-89284fff0373 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:44.319517 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 149/291] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:44 2022] GET /identity/v3/users/03f24986305c4ed2913113729807ebec => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:44.321173 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dabfb3-c690-44d6-b3dc-1b34cc51ef66 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:44.439364 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dabfb3-c690-44d6-b3dc-1b34cc51ef66 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:44.475067 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:44.485431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676371e1-c6ff-42e4-a6b2-152c38853515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:44.486076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676371e1-c6ff-42e4-a6b2-152c38853515 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:44.486358 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676371e1-c6ff-42e4-a6b2-152c38853515 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:44.487357 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676371e1-c6ff-42e4-a6b2-152c38853515 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:44.580084 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676371e1-c6ff-42e4-a6b2-152c38853515 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:44.581542 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676371e1-c6ff-42e4-a6b2-152c38853515 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:44.684468 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 143/292] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:29:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:29:44.690309 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51606bac-0ff2-48fd-82e0-89284fff0373 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:44.811164 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51606bac-0ff2-48fd-82e0-89284fff0373 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:44.851948 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:44.858219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e180ad-64a1-435e-93f0-1bb14f232c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:44.859129 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e180ad-64a1-435e-93f0-1bb14f232c80 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:44.859431 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e180ad-64a1-435e-93f0-1bb14f232c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:44.859705 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e180ad-64a1-435e-93f0-1bb14f232c80 None admin] PATH_INFO: `/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:44.860866 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e180ad-64a1-435e-93f0-1bb14f232c80 None admin] RBAC: Authorizing `identity:get_project(project_id=5dbf1af61d63414cbf4c33bdc412a9d6)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:44.862302 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e180ad-64a1-435e-93f0-1bb14f232c80 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:44.869113 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 150/293] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:44 2022] GET /identity/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:44.870597 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676371e1-c6ff-42e4-a6b2-152c38853515 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:44.901931 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:44.954199 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44049ea0-2acd-49c5-9480-80640ec9e6ed None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:44.954491 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44049ea0-2acd-49c5-9480-80640ec9e6ed None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:44.954671 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44049ea0-2acd-49c5-9480-80640ec9e6ed None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:44.956054 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 151/294] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:44.965588 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aceee7a-8bba-4790-b70a-6c7e32deaa77 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:44.965860 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aceee7a-8bba-4790-b70a-6c7e32deaa77 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:44.966139 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aceee7a-8bba-4790-b70a-6c7e32deaa77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:44.994376 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-0aceee7a-8bba-4790-b70a-6c7e32deaa77 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:44.995592 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0aceee7a-8bba-4790-b70a-6c7e32deaa77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:45.010287 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676371e1-c6ff-42e4-a6b2-152c38853515 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:45.058203 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170371c-c09f-4422-87aa-0bd59186696c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:45.059105 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170371c-c09f-4422-87aa-0bd59186696c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:45.059503 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170371c-c09f-4422-87aa-0bd59186696c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:45.059874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170371c-c09f-4422-87aa-0bd59186696c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:45.060869 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0170371c-c09f-4422-87aa-0bd59186696c None admin] RBAC: Adding query filter params (role.id=e033961a9363456a8510c34414dd839e, scope.project.id=5dbf1af61d63414cbf4c33bdc412a9d6, user.id=03f24986305c4ed2913113729807ebec) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:45.061189 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0170371c-c09f-4422-87aa-0bd59186696c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:45.062880 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0170371c-c09f-4422-87aa-0bd59186696c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:45.069123 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 144/295] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:29:44 2022] GET /identity/v3/role_assignments?user.id=03f24986305c4ed2913113729807ebec&scope.project.id=5dbf1af61d63414cbf4c33bdc412a9d6&role.id=e033961a9363456a8510c34414dd839e => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:45.070350 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced5124-1fe5-4123-9211-43f6f6eba017 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:45.070677 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced5124-1fe5-4123-9211-43f6f6eba017 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:45.071082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced5124-1fe5-4123-9211-43f6f6eba017 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:45.072642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 145/296] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:45.073775 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e8d42-54c3-4cfc-bcb4-a1571bd53b73 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:45.074056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e8d42-54c3-4cfc-bcb4-a1571bd53b73 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:45.074353 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e8d42-54c3-4cfc-bcb4-a1571bd53b73 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:45.075735 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 146/297] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:45.081000 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072e47f-5032-4de5-a9fc-906f91d46568 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:45.081305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072e47f-5032-4de5-a9fc-906f91d46568 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:45.081550 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072e47f-5032-4de5-a9fc-906f91d46568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:45.102634 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a072e47f-5032-4de5-a9fc-906f91d46568 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:45.103042 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 152/298] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:45.104686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265eabb1-ca2f-4fe0-9f89-6122c7ff02cb None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:45.105201 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265eabb1-ca2f-4fe0-9f89-6122c7ff02cb None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:45.105764 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265eabb1-ca2f-4fe0-9f89-6122c7ff02cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:45.112144 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a072e47f-5032-4de5-a9fc-906f91d46568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:45.136051 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-265eabb1-ca2f-4fe0-9f89-6122c7ff02cb None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:45.137542 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-265eabb1-ca2f-4fe0-9f89-6122c7ff02cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:45.201684 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 147/299] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:45.202830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072e47f-5032-4de5-a9fc-906f91d46568 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:45.243156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:45.261615 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 153/300] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:45.262702 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265eabb1-ca2f-4fe0-9f89-6122c7ff02cb None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:45.344570 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:45.358989 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072e47f-5032-4de5-a9fc-906f91d46568 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:45.389568 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265eabb1-ca2f-4fe0-9f89-6122c7ff02cb None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:45.395081 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd70f7b-7d57-4ed4-be1a-fc097222fabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:45.395751 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd70f7b-7d57-4ed4-be1a-fc097222fabe None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:45.396032 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd70f7b-7d57-4ed4-be1a-fc097222fabe None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:45.396341 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd70f7b-7d57-4ed4-be1a-fc097222fabe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:45.408119 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd70f7b-7d57-4ed4-be1a-fc097222fabe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:45.413386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-4dd70f7b-7d57-4ed4-be1a-fc097222fabe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:45.413386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:45.413386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:45.413386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:45.413386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:45.413386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:45.413386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:45.413386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:45.413386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:45.413386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:45.413386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:45.413386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:45.413386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:45.413386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:45.413386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:45.413386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:45.414243 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:45.414243 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:45.414243 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:45.414243 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:45.414243 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:45.414243 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:45.414243 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:45.414243 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:45.414243 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:45.414243 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:45.414243 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:45.414243 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:45.414243 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:45.414243 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:45.414243 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:45.414243 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:45.415079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:45.415079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:45.415079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:45.415079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:45.415079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:45.415079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:45.415079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:45.415079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:45.415079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:45.415079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:45.415079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:45.415079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:29:45.415079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 148/301] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:45.415761 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd70f7b-7d57-4ed4-be1a-fc097222fabe None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:45.434447 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667b22a-8627-43cc-ac9b-47cdafd9db49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:45.435703 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667b22a-8627-43cc-ac9b-47cdafd9db49 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:45.436140 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667b22a-8627-43cc-ac9b-47cdafd9db49 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:45.436562 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667b22a-8627-43cc-ac9b-47cdafd9db49 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:45.453441 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d667b22a-8627-43cc-ac9b-47cdafd9db49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:45.459167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-d667b22a-8627-43cc-ac9b-47cdafd9db49 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:29:45.459167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:45.459167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:45.459167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:45.459167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:45.459167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:45.459167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:45.459167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:45.459167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:45.459167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:45.459167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:45.459167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:45.459167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:45.459167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:45.459167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:45.459167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:45.460263 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:45.460263 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:45.460263 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:45.460263 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:45.460263 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:45.460263 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:45.460263 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:45.460263 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:45.460263 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:45.460263 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:45.460263 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:45.460263 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:45.460263 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:45.460263 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:45.460263 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:45.460263 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:45.461230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:45.461230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:45.461230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:45.461230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:45.461230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:45.461230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:45.461230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:45.461230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:45.461230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:45.461230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:45.461230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:29:45.461230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:29:45.463884 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 154/302] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:45.464812 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667b22a-8627-43cc-ac9b-47cdafd9db49 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:45.509598 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:45.526124 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd70f7b-7d57-4ed4-be1a-fc097222fabe None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:45.563318 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd3563-90ce-4fd4-9bcd-5261713b8330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:45.564033 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd3563-90ce-4fd4-9bcd-5261713b8330 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:45.564319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd3563-90ce-4fd4-9bcd-5261713b8330 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:45.564593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd3563-90ce-4fd4-9bcd-5261713b8330 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:45.565748 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fd3563-90ce-4fd4-9bcd-5261713b8330 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:45.567184 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fd3563-90ce-4fd4-9bcd-5261713b8330 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:45.568804 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 149/303] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:29:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:45.569557 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd3563-90ce-4fd4-9bcd-5261713b8330 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:45.585282 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667b22a-8627-43cc-ac9b-47cdafd9db49 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:45.607624 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:45.624523 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2b7e4-2a20-4592-950f-001e239e20b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:45.625170 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2b7e4-2a20-4592-950f-001e239e20b1 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:45.625450 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2b7e4-2a20-4592-950f-001e239e20b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:45.625719 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2b7e4-2a20-4592-950f-001e239e20b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:45.626425 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2b7e4-2a20-4592-950f-001e239e20b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:45.626728 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2b7e4-2a20-4592-950f-001e239e20b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:45.628116 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2b7e4-2a20-4592-950f-001e239e20b1 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:45.636695 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 155/304] 10.222.0.97 () {56 vars in 1101 bytes} [Tue Jul 26 14:29:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:45.637817 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2b7e4-2a20-4592-950f-001e239e20b1 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:45.678931 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd3563-90ce-4fd4-9bcd-5261713b8330 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:45.714709 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b2fad-32a8-4f9d-b7ef-7806c762970a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:45.715936 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b2fad-32a8-4f9d-b7ef-7806c762970a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:45.716198 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b2fad-32a8-4f9d-b7ef-7806c762970a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:45.716469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b2fad-32a8-4f9d-b7ef-7806c762970a None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:45.717183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b2fad-32a8-4f9d-b7ef-7806c762970a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:45.717497 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b2fad-32a8-4f9d-b7ef-7806c762970a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:45.718843 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b2fad-32a8-4f9d-b7ef-7806c762970a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:45.731806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 150/305] 10.222.0.97 () {56 vars in 1101 bytes} [Tue Jul 26 14:29:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:45.731806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b2fad-32a8-4f9d-b7ef-7806c762970a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:45.747555 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2b7e4-2a20-4592-950f-001e239e20b1 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:45.748210 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:45.781808 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9173ca-6a89-469c-ad43-22c09fcd4813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:45.782775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9173ca-6a89-469c-ad43-22c09fcd4813 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:45.783228 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9173ca-6a89-469c-ad43-22c09fcd4813 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:45.783695 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9173ca-6a89-469c-ad43-22c09fcd4813 None admin] PATH_INFO: `/v3/groups` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:45.784532 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9173ca-6a89-469c-ad43-22c09fcd4813 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:45.786170 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9173ca-6a89-469c-ad43-22c09fcd4813 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:45.831303 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b2fad-32a8-4f9d-b7ef-7806c762970a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:45.864892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3273da-2d36-4279-96e7-34950b7731e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:45.865529 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3273da-2d36-4279-96e7-34950b7731e1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:45.865811 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3273da-2d36-4279-96e7-34950b7731e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:45.866087 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3273da-2d36-4279-96e7-34950b7731e1 None admin] PATH_INFO: `/v3/users/07a7f8d10eb44197a4335c6f76dc9c9b` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:45.867303 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3273da-2d36-4279-96e7-34950b7731e1 None admin] RBAC: Authorizing `identity:get_user(user_id=07a7f8d10eb44197a4335c6f76dc9c9b)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:45.868773 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3273da-2d36-4279-96e7-34950b7731e1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:45.870408 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 151/306] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:45 2022] GET /identity/v3/users/07a7f8d10eb44197a4335c6f76dc9c9b => generated 293 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:45.871188 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3273da-2d36-4279-96e7-34950b7731e1 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:45.953112 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 156/307] 10.222.0.97 () {60 vars in 1134 bytes} [Tue Jul 26 14:29:45 2022] POST /identity/v3/groups => generated 229 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:29:45.954424 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9173ca-6a89-469c-ad43-22c09fcd4813 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:45.971596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3273da-2d36-4279-96e7-34950b7731e1 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:45.977064 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:46.011901 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79c31f-4d09-4ab6-8bfe-ee85fb9ff91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:46.012575 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79c31f-4d09-4ab6-8bfe-ee85fb9ff91b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:46.012861 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79c31f-4d09-4ab6-8bfe-ee85fb9ff91b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:46.013350 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79c31f-4d09-4ab6-8bfe-ee85fb9ff91b None admin] PATH_INFO: `/v3/users/5a0514276028451a8ad5e80aba582f37` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:46.024628 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:46.042412 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79c31f-4d09-4ab6-8bfe-ee85fb9ff91b None admin] RBAC: Authorizing `identity:get_user(user_id=5a0514276028451a8ad5e80aba582f37)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:46.042412 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79c31f-4d09-4ab6-8bfe-ee85fb9ff91b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:46.043902 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 152/308] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:45 2022] GET /identity/v3/users/5a0514276028451a8ad5e80aba582f37 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:46.045469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79c31f-4d09-4ab6-8bfe-ee85fb9ff91b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:46.119527 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9173ca-6a89-469c-ad43-22c09fcd4813 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:46.167631 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7aafe-a2ea-440f-a6d9-1aceba0ae9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:46.168734 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7aafe-a2ea-440f-a6d9-1aceba0ae9c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:46.169192 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7aafe-a2ea-440f-a6d9-1aceba0ae9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:46.169629 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7aafe-a2ea-440f-a6d9-1aceba0ae9c0 None admin] PATH_INFO: `/v3/system/users/07a7f8d10eb44197a4335c6f76dc9c9b/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:46.172016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7aafe-a2ea-440f-a6d9-1aceba0ae9c0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=07a7f8d10eb44197a4335c6f76dc9c9b, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:46.180081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7aafe-a2ea-440f-a6d9-1aceba0ae9c0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:46.184016 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79c31f-4d09-4ab6-8bfe-ee85fb9ff91b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:46.221146 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fc8d3-86eb-4a6b-b3b0-0c62f28a713f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:46.221765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fc8d3-86eb-4a6b-b3b0-0c62f28a713f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:46.222054 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fc8d3-86eb-4a6b-b3b0-0c62f28a713f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:46.222332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fc8d3-86eb-4a6b-b3b0-0c62f28a713f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:46.223129 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0fc8d3-86eb-4a6b-b3b0-0c62f28a713f None admin] RBAC: Adding query filter params (role.id=2b922104c92a4cbc9432fbce1c9bda1e, user.id=5a0514276028451a8ad5e80aba582f37, scope.system=all) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:46.223473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0fc8d3-86eb-4a6b-b3b0-0c62f28a713f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:46.224964 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0fc8d3-86eb-4a6b-b3b0-0c62f28a713f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:46.230300 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 153/309] 10.222.0.97 () {56 vars in 1308 bytes} [Tue Jul 26 14:29:46 2022] GET /identity/v3/role_assignments?user.id=5a0514276028451a8ad5e80aba582f37&scope.system=all&role.id=2b922104c92a4cbc9432fbce1c9bda1e => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:46.267319 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 157/310] 10.222.0.97 () {58 vars in 1334 bytes} [Tue Jul 26 14:29:45 2022] PUT /identity/v3/system/users/07a7f8d10eb44197a4335c6f76dc9c9b/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:29:46.371022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afc107-07e4-416a-836d-aa14a06cd3a9 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:46.371801 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afc107-07e4-416a-836d-aa14a06cd3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:46.372294 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afc107-07e4-416a-836d-aa14a06cd3a9 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:46.374328 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 154/311] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:46.380474 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b8361-9c0d-4371-9be8-1d520673e768 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:46.381129 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b8361-9c0d-4371-9be8-1d520673e768 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:46.381622 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b8361-9c0d-4371-9be8-1d520673e768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:46.403696 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b13b8361-9c0d-4371-9be8-1d520673e768 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:46.405320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b13b8361-9c0d-4371-9be8-1d520673e768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:46.482733 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 158/312] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:46.506220 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4afc107-07e4-416a-836d-aa14a06cd3a9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:46.511352 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:46.615328 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4afc107-07e4-416a-836d-aa14a06cd3a9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:46.651288 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c31ed47-72b1-4a34-b943-d09322346e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:46.652178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31ed47-72b1-4a34-b943-d09322346e9f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:46.653153 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31ed47-72b1-4a34-b943-d09322346e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:46.653588 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31ed47-72b1-4a34-b943-d09322346e9f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:46.664598 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c31ed47-72b1-4a34-b943-d09322346e9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:46.671646 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-4c31ed47-72b1-4a34-b943-d09322346e9f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:29:46.671646 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:46.671646 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:46.671646 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:46.671646 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:46.671646 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:46.671646 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:46.671646 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:46.671646 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:46.671646 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:46.671646 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:46.671646 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:46.671646 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:46.671646 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:46.671646 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:46.671646 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:46.672761 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:46.672761 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:46.672761 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:46.672761 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:46.672761 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:46.672761 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:46.672761 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:46.672761 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:46.672761 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:46.672761 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:46.672761 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:46.672761 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:46.672761 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:46.672761 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:46.672761 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:46.672761 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:46.673788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:46.673788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:46.673788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:46.673788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:46.673788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:46.673788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:46.673788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:46.673788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:46.673788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:46.673788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:46.673788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:29:46.673788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:29:46.676060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 155/313] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:46.687461 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b8361-9c0d-4371-9be8-1d520673e768 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:46.721986 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:46.794652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b8361-9c0d-4371-9be8-1d520673e768 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:46.830568 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d897212-c38f-4abc-9295-03e04743be88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:46.831195 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d897212-c38f-4abc-9295-03e04743be88 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:46.831504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d897212-c38f-4abc-9295-03e04743be88 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:46.831782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d897212-c38f-4abc-9295-03e04743be88 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:46.832495 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d897212-c38f-4abc-9295-03e04743be88 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:46.832799 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d897212-c38f-4abc-9295-03e04743be88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:46.834181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d897212-c38f-4abc-9295-03e04743be88 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:46.843607 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 159/314] 10.222.0.97 () {56 vars in 1101 bytes} [Tue Jul 26 14:29:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:46.848848 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c31ed47-72b1-4a34-b943-d09322346e9f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:46.915065 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:46.953152 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c31ed47-72b1-4a34-b943-d09322346e9f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:46.991306 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a94bc66-8f32-46db-a207-10264a0d6cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:46.992363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94bc66-8f32-46db-a207-10264a0d6cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:46.992861 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94bc66-8f32-46db-a207-10264a0d6cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:46.993352 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94bc66-8f32-46db-a207-10264a0d6cb3 None admin] PATH_INFO: `/v3/users/822f3e0568314a2b8663a80c73fc8830` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:46.994738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a94bc66-8f32-46db-a207-10264a0d6cb3 None admin] RBAC: Authorizing `identity:get_user(user_id=822f3e0568314a2b8663a80c73fc8830)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:46.996455 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a94bc66-8f32-46db-a207-10264a0d6cb3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:46.998569 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 156/315] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:46 2022] GET /identity/v3/users/822f3e0568314a2b8663a80c73fc8830 => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:47.006760 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d897212-c38f-4abc-9295-03e04743be88 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:47.115152 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:47.121067 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d897212-c38f-4abc-9295-03e04743be88 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:47.156025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d23eb-655f-4d59-80bd-f53f427d72ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:47.156999 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d23eb-655f-4d59-80bd-f53f427d72ea None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:47.157287 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d23eb-655f-4d59-80bd-f53f427d72ea None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:47.157565 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d23eb-655f-4d59-80bd-f53f427d72ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:47.229907 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7d23eb-655f-4d59-80bd-f53f427d72ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:47.231350 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7d23eb-655f-4d59-80bd-f53f427d72ea None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:47.326261 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:47.333132 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 160/316] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:29:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:29:47.338498 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a94bc66-8f32-46db-a207-10264a0d6cb3 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:47.445329 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a94bc66-8f32-46db-a207-10264a0d6cb3 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:47.481343 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:47.487025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f878ce-583e-4f34-85e6-a655bae0609e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:47.494913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f878ce-583e-4f34-85e6-a655bae0609e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:47.494913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f878ce-583e-4f34-85e6-a655bae0609e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:47.494913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f878ce-583e-4f34-85e6-a655bae0609e None admin] PATH_INFO: `/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:47.494913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f878ce-583e-4f34-85e6-a655bae0609e None admin] RBAC: Authorizing `identity:get_project(project_id=5dbf1af61d63414cbf4c33bdc412a9d6)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:47.494913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f878ce-583e-4f34-85e6-a655bae0609e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:47.494913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 157/317] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:47 2022] GET /identity/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:47.503443 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d23eb-655f-4d59-80bd-f53f427d72ea None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:47.601594 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d23eb-655f-4d59-80bd-f53f427d72ea None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:47.636859 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75fbeb0-a60f-4125-b275-3c59e5a5c0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:47.637483 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fbeb0-a60f-4125-b275-3c59e5a5c0c3 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:47.637766 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fbeb0-a60f-4125-b275-3c59e5a5c0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:47.638042 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fbeb0-a60f-4125-b275-3c59e5a5c0c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:47.638832 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75fbeb0-a60f-4125-b275-3c59e5a5c0c3 None admin] RBAC: Adding query filter params (role.id=2b922104c92a4cbc9432fbce1c9bda1e, scope.project.id=5dbf1af61d63414cbf4c33bdc412a9d6, user.id=822f3e0568314a2b8663a80c73fc8830) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:47.639142 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75fbeb0-a60f-4125-b275-3c59e5a5c0c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:47.640619 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75fbeb0-a60f-4125-b275-3c59e5a5c0c3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:47.646966 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 161/318] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:29:47 2022] GET /identity/v3/role_assignments?user.id=822f3e0568314a2b8663a80c73fc8830&scope.project.id=5dbf1af61d63414cbf4c33bdc412a9d6&role.id=2b922104c92a4cbc9432fbce1c9bda1e => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:48.182882 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe61a3-a61b-4975-99ea-780b29b20f61 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:48.183258 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe61a3-a61b-4975-99ea-780b29b20f61 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:48.184460 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe61a3-a61b-4975-99ea-780b29b20f61 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:48.185962 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 158/319] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:48.191247 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38187913-44f7-459b-a361-78ea6c759556 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:48.191563 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38187913-44f7-459b-a361-78ea6c759556 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:48.191843 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38187913-44f7-459b-a361-78ea6c759556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:48.210241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-38187913-44f7-459b-a361-78ea6c759556 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:48.211447 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-38187913-44f7-459b-a361-78ea6c759556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:48.270992 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:48.279910 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204dfb1d-de56-4b10-b29e-4bec63eadeaf None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:48.280278 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204dfb1d-de56-4b10-b29e-4bec63eadeaf None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:48.280697 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204dfb1d-de56-4b10-b29e-4bec63eadeaf None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:48.282427 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 159/320] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:29:48.285868 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 162/321] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:48.287424 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378b842-ce0f-4fd8-bfce-234a55a4bab5 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:48.287710 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378b842-ce0f-4fd8-bfce-234a55a4bab5 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:48.287995 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378b842-ce0f-4fd8-bfce-234a55a4bab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:48.305963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-9378b842-ce0f-4fd8-bfce-234a55a4bab5 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:48.306606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38187913-44f7-459b-a361-78ea6c759556 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:48.307085 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9378b842-ce0f-4fd8-bfce-234a55a4bab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:48.380646 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 160/322] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:48.389423 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f062269-9213-417e-b812-ede96a7f4793 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:48.389715 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f062269-9213-417e-b812-ede96a7f4793 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:48.389994 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f062269-9213-417e-b812-ede96a7f4793 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:48.391439 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 161/323] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:29:48.396172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772b390-15e7-4327-9a8e-1e7755e55198 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:48.396460 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772b390-15e7-4327-9a8e-1e7755e55198 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:48.396735 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772b390-15e7-4327-9a8e-1e7755e55198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:48.412351 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38187913-44f7-459b-a361-78ea6c759556 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:48.414211 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-5772b390-15e7-4327-9a8e-1e7755e55198 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:48.415390 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5772b390-15e7-4327-9a8e-1e7755e55198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:48.446112 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1379f5b-ea70-4f92-af38-d42562953638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:48.446757 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1379f5b-ea70-4f92-af38-d42562953638 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:48.447042 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1379f5b-ea70-4f92-af38-d42562953638 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:48.447319 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1379f5b-ea70-4f92-af38-d42562953638 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:48.448490 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1379f5b-ea70-4f92-af38-d42562953638 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:48.450476 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1379f5b-ea70-4f92-af38-d42562953638 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:48.452091 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 163/324] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:29:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:48.452849 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1379f5b-ea70-4f92-af38-d42562953638 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:48.482253 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 162/325] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:29:48.482995 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772b390-15e7-4327-9a8e-1e7755e55198 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:48.512350 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:48.541390 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:48.552734 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1379f5b-ea70-4f92-af38-d42562953638 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:48.582130 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772b390-15e7-4327-9a8e-1e7755e55198 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:48.587010 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef46bf8-6bc7-472f-bb13-8151c7f9f866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:48.587702 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef46bf8-6bc7-472f-bb13-8151c7f9f866 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:48.587978 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef46bf8-6bc7-472f-bb13-8151c7f9f866 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:48.588256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef46bf8-6bc7-472f-bb13-8151c7f9f866 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:48.599374 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef46bf8-6bc7-472f-bb13-8151c7f9f866 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:48.604389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-3ef46bf8-6bc7-472f-bb13-8151c7f9f866 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:29:48.604389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:48.604389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:48.604389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:48.604389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:48.604389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:48.604389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:48.604389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:48.604389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:48.604389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:48.604389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:48.604389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:48.604389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:48.604389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:48.604389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:48.604389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:48.605238 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:48.605238 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:48.605238 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:48.605238 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:48.605238 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:48.605238 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:48.605238 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:48.605238 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:48.605238 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:48.605238 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:48.605238 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:48.605238 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:48.605238 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:48.605238 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:48.605238 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:48.605238 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:48.606645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:48.606645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:48.606645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:48.606645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:48.606645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:48.606645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:48.606645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:48.606645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:48.606645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:48.606645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:48.606645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:29:48.606645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:29:48.606645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 164/326] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:48.607293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef46bf8-6bc7-472f-bb13-8151c7f9f866 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:48.620166 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f888df-565f-4256-b756-707a59a18289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:48.620781 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f888df-565f-4256-b756-707a59a18289 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:48.621129 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f888df-565f-4256-b756-707a59a18289 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:48.621403 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f888df-565f-4256-b756-707a59a18289 None admin] PATH_INFO: `/v3/groups` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:48.622053 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f888df-565f-4256-b756-707a59a18289 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:48.623503 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f888df-565f-4256-b756-707a59a18289 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:48.706026 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef46bf8-6bc7-472f-bb13-8151c7f9f866 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:48.722537 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 163/327] 10.222.0.97 () {60 vars in 1134 bytes} [Tue Jul 26 14:29:48 2022] POST /identity/v3/groups => generated 226 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:29:48.731247 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f888df-565f-4256-b756-707a59a18289 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:48.731459 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:48.741696 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e3117-29d2-4290-be00-771e513844b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:48.742374 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e3117-29d2-4290-be00-771e513844b0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:48.742661 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e3117-29d2-4290-be00-771e513844b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:48.742935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e3117-29d2-4290-be00-771e513844b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:48.755314 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e3117-29d2-4290-be00-771e513844b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:48.760593 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-8d2e3117-29d2-4290-be00-771e513844b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:48.760593 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:48.760593 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:48.760593 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:48.760593 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:48.760593 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:48.760593 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:48.760593 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:48.760593 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:48.760593 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:48.760593 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:48.760593 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:48.760593 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:48.760593 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:48.760593 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:48.760593 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:48.761578 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:48.761578 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:48.761578 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:48.761578 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:48.761578 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:48.761578 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:48.761578 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:48.761578 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:48.761578 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:48.761578 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:48.761578 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:48.761578 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:48.761578 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:48.761578 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:48.761578 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:48.761578 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:48.762462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:48.762462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:48.762462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:48.762462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:48.762462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:48.762462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:48.762462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:48.762462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:48.762462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:48.762462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:48.762462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:48.762462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:29:48.762462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 165/328] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:48.769725 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e3117-29d2-4290-be00-771e513844b0 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:48.862750 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f888df-565f-4256-b756-707a59a18289 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:48.884138 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:48.888445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e3117-29d2-4290-be00-771e513844b0 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:48.899738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d3891-a633-4d95-89b6-0beec0f029b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:48.900440 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d3891-a633-4d95-89b6-0beec0f029b7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:48.901392 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d3891-a633-4d95-89b6-0beec0f029b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:48.901392 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d3891-a633-4d95-89b6-0beec0f029b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:48.902977 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d3891-a633-4d95-89b6-0beec0f029b7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:48.902977 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d3891-a633-4d95-89b6-0beec0f029b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:48.904782 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d3891-a633-4d95-89b6-0beec0f029b7 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:48.914228 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 164/329] 10.222.0.97 () {56 vars in 1101 bytes} [Tue Jul 26 14:29:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:48.919998 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d3891-a633-4d95-89b6-0beec0f029b7 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:48.929884 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff749b1-1059-48d6-bd14-a68fd2b5bf35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:48.930504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff749b1-1059-48d6-bd14-a68fd2b5bf35 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:48.930787 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff749b1-1059-48d6-bd14-a68fd2b5bf35 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:48.931254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff749b1-1059-48d6-bd14-a68fd2b5bf35 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:48.931862 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff749b1-1059-48d6-bd14-a68fd2b5bf35 None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:48.932165 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff749b1-1059-48d6-bd14-a68fd2b5bf35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:48.934230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff749b1-1059-48d6-bd14-a68fd2b5bf35 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:48.942761 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 166/330] 10.222.0.97 () {56 vars in 1101 bytes} [Tue Jul 26 14:29:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:48.948567 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff749b1-1059-48d6-bd14-a68fd2b5bf35 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:49.002000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:49.043834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d3891-a633-4d95-89b6-0beec0f029b7 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:49.085902 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff749b1-1059-48d6-bd14-a68fd2b5bf35 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:49.095074 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03275114-85c8-456c-9ec8-b14409f8e9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:49.095768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03275114-85c8-456c-9ec8-b14409f8e9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:49.096056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03275114-85c8-456c-9ec8-b14409f8e9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:49.096332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03275114-85c8-456c-9ec8-b14409f8e9d0 None admin] PATH_INFO: `/v3/users/5a0514276028451a8ad5e80aba582f37` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:49.097545 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03275114-85c8-456c-9ec8-b14409f8e9d0 None admin] RBAC: Authorizing `identity:get_user(user_id=5a0514276028451a8ad5e80aba582f37)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:49.099062 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03275114-85c8-456c-9ec8-b14409f8e9d0 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:49.101019 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 165/331] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:48 2022] GET /identity/v3/users/5a0514276028451a8ad5e80aba582f37 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:49.106703 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03275114-85c8-456c-9ec8-b14409f8e9d0 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:49.138969 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b32818-a746-4ae9-a399-5997304812f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:49.140566 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b32818-a746-4ae9-a399-5997304812f4 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:49.141075 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b32818-a746-4ae9-a399-5997304812f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:49.141782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b32818-a746-4ae9-a399-5997304812f4 None admin] PATH_INFO: `/v3/users/07a7f8d10eb44197a4335c6f76dc9c9b` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:49.144801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b32818-a746-4ae9-a399-5997304812f4 None admin] RBAC: Authorizing `identity:get_user(user_id=07a7f8d10eb44197a4335c6f76dc9c9b)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:49.145981 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b32818-a746-4ae9-a399-5997304812f4 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:49.149426 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 167/332] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:48 2022] GET /identity/v3/users/07a7f8d10eb44197a4335c6f76dc9c9b => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:49.155738 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b32818-a746-4ae9-a399-5997304812f4 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:49.167169 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:49.221167 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03275114-85c8-456c-9ec8-b14409f8e9d0 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:49.265709 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4033a5-9195-49a6-ac23-da53dc85d836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:49.266651 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4033a5-9195-49a6-ac23-da53dc85d836 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:49.267091 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4033a5-9195-49a6-ac23-da53dc85d836 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:49.267843 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4033a5-9195-49a6-ac23-da53dc85d836 None admin] PATH_INFO: `/v3/system/users/5a0514276028451a8ad5e80aba582f37/roles/2b922104c92a4cbc9432fbce1c9bda1e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:49.269557 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4033a5-9195-49a6-ac23-da53dc85d836 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5a0514276028451a8ad5e80aba582f37, role_id=2b922104c92a4cbc9432fbce1c9bda1e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:49.271186 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4033a5-9195-49a6-ac23-da53dc85d836 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:49.293540 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b32818-a746-4ae9-a399-5997304812f4 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:49.305356 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:49.333778 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072e86e-f8e2-4c4d-b75b-693e66ea0a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:49.334694 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072e86e-f8e2-4c4d-b75b-693e66ea0a0c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:49.335152 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072e86e-f8e2-4c4d-b75b-693e66ea0a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:49.335603 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072e86e-f8e2-4c4d-b75b-693e66ea0a0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:49.336542 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e072e86e-f8e2-4c4d-b75b-693e66ea0a0c None admin] RBAC: Adding query filter params (role.id=e033961a9363456a8510c34414dd839e, user.id=07a7f8d10eb44197a4335c6f76dc9c9b, scope.system=all) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:49.336999 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e072e86e-f8e2-4c4d-b75b-693e66ea0a0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:49.338585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e072e86e-f8e2-4c4d-b75b-693e66ea0a0c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:49.351973 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 168/333] 10.222.0.97 () {56 vars in 1308 bytes} [Tue Jul 26 14:29:49 2022] GET /identity/v3/role_assignments?user.id=07a7f8d10eb44197a4335c6f76dc9c9b&scope.system=all&role.id=e033961a9363456a8510c34414dd839e => generated 510 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:49.354667 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 166/334] 10.222.0.97 () {58 vars in 1334 bytes} [Tue Jul 26 14:29:49 2022] PUT /identity/v3/system/users/5a0514276028451a8ad5e80aba582f37/roles/2b922104c92a4cbc9432fbce1c9bda1e => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:29:50.995789 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42afb8-37c2-4ab6-ab27-5b721d7354a7 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:50.996417 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42afb8-37c2-4ab6-ab27-5b721d7354a7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:50.996417 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42afb8-37c2-4ab6-ab27-5b721d7354a7 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:50.997993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 167/335] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:51.003420 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e425ccc-b42b-4b40-841d-e0a4803f33d9 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:51.003798 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e425ccc-b42b-4b40-841d-e0a4803f33d9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:51.003993 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e425ccc-b42b-4b40-841d-e0a4803f33d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:51.028397 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6e425ccc-b42b-4b40-841d-e0a4803f33d9 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:51.029580 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6e425ccc-b42b-4b40-841d-e0a4803f33d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:51.128646 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 169/336] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:51.163191 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba42afb8-37c2-4ab6-ab27-5b721d7354a7 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:51.290405 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba42afb8-37c2-4ab6-ab27-5b721d7354a7 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:51.304904 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:51.326475 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e473fa7-624c-47d5-822e-c8fc3dbde8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:51.327122 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e473fa7-624c-47d5-822e-c8fc3dbde8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:51.327510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e473fa7-624c-47d5-822e-c8fc3dbde8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:51.327787 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e473fa7-624c-47d5-822e-c8fc3dbde8c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:51.343368 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e473fa7-624c-47d5-822e-c8fc3dbde8c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:51.352155 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-8e473fa7-624c-47d5-822e-c8fc3dbde8c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:51.352155 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:51.352155 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:51.352155 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:51.352155 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:51.352155 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:51.352155 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:51.352155 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:51.352155 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:51.352155 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:51.352155 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:51.352155 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:51.352155 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:51.352155 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:51.352155 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:51.352155 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:51.353010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:51.353010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:51.353010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:51.353010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:51.353010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:51.353010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:51.353010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:51.353010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:51.353010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:51.353010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:51.353010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:51.353010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:51.353010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:51.353010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:51.353010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:51.353010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:51.353785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:51.353785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:51.353785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:51.353785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:51.353785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:51.353785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:51.353785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:51.353785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:51.353785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:51.353785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:51.353785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:51.353785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:29:51.353785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 168/337] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:51.360455 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e425ccc-b42b-4b40-841d-e0a4803f33d9 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:51.484848 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:51.492097 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e425ccc-b42b-4b40-841d-e0a4803f33d9 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:51.533156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4112b65-9be9-4d5d-bf23-f431f163ce30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:51.533801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4112b65-9be9-4d5d-bf23-f431f163ce30 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:51.534087 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4112b65-9be9-4d5d-bf23-f431f163ce30 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:51.534366 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4112b65-9be9-4d5d-bf23-f431f163ce30 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:51.535108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4112b65-9be9-4d5d-bf23-f431f163ce30 None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:51.535433 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4112b65-9be9-4d5d-bf23-f431f163ce30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:51.536837 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4112b65-9be9-4d5d-bf23-f431f163ce30 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:51.546660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 170/338] 10.222.0.97 () {56 vars in 1101 bytes} [Tue Jul 26 14:29:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:51.552356 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e473fa7-624c-47d5-822e-c8fc3dbde8c2 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:51.648454 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93482d1-891e-4f5d-8c86-fa5d836183fb None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:51.648809 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93482d1-891e-4f5d-8c86-fa5d836183fb None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:51.649014 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93482d1-891e-4f5d-8c86-fa5d836183fb None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:51.650399 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 171/339] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:51.656229 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e4e78-974b-488b-b37c-64d6fa7529e6 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:51.656642 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e4e78-974b-488b-b37c-64d6fa7529e6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:51.656922 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e4e78-974b-488b-b37c-64d6fa7529e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:51.685460 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-651e4e78-974b-488b-b37c-64d6fa7529e6 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:51.686806 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-651e4e78-974b-488b-b37c-64d6fa7529e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:51.694376 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e473fa7-624c-47d5-822e-c8fc3dbde8c2 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:51.785894 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39850408-f8d1-42c6-abd0-ded7f6316b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:51.786522 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39850408-f8d1-42c6-abd0-ded7f6316b88 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:51.786802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39850408-f8d1-42c6-abd0-ded7f6316b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:51.787078 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39850408-f8d1-42c6-abd0-ded7f6316b88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:51.788927 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 172/340] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:29:51.832967 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e4e78-974b-488b-b37c-64d6fa7529e6 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:51.859815 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:51.918277 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39850408-f8d1-42c6-abd0-ded7f6316b88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:51.919804 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39850408-f8d1-42c6-abd0-ded7f6316b88 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:52.020817 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:52.031880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e4e78-974b-488b-b37c-64d6fa7529e6 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:52.037618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 169/341] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:29:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:29:52.043583 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2dea5-4f0c-4bff-8260-8531d396c8e1 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:52.045490 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2dea5-4f0c-4bff-8260-8531d396c8e1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:52.046047 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2dea5-4f0c-4bff-8260-8531d396c8e1 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:52.047869 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 170/342] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:52.048826 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2dea5-4f0c-4bff-8260-8531d396c8e1 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:52.083885 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb141c01-1ebf-485f-83a9-b7e54f874382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:52.084464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb141c01-1ebf-485f-83a9-b7e54f874382 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:52.084741 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb141c01-1ebf-485f-83a9-b7e54f874382 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:52.085048 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb141c01-1ebf-485f-83a9-b7e54f874382 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:52.108122 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb141c01-1ebf-485f-83a9-b7e54f874382 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:52.113109 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-eb141c01-1ebf-485f-83a9-b7e54f874382 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:29:52.113109 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:52.113109 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:52.113109 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:52.113109 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:52.113109 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:52.113109 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:52.113109 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:52.113109 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:52.113109 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:52.113109 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:52.113109 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:52.113109 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:52.113109 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:52.113109 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:52.113109 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:52.113970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:52.113970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:52.113970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:52.113970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:52.113970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:52.113970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:52.113970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:52.113970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:52.113970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:52.113970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:52.113970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:52.113970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:52.113970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:52.113970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:52.113970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:52.113970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:52.114741 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:52.114741 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:52.114741 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:52.114741 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:52.114741 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:52.114741 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:52.114741 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:52.114741 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:52.114741 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:52.114741 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:52.114741 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:29:52.114741 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:29:52.124632 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 173/343] 10.222.0.97 () {56 vars in 1114 bytes} [Tue Jul 26 14:29:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:52.126085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182baea-d081-4e70-986b-eed90fc8948e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:52.126375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182baea-d081-4e70-986b-eed90fc8948e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:52.126650 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182baea-d081-4e70-986b-eed90fc8948e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:52.161332 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-0182baea-d081-4e70-986b-eed90fc8948e None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:52.163078 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0182baea-d081-4e70-986b-eed90fc8948e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:52.173146 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:52.229444 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2dea5-4f0c-4bff-8260-8531d396c8e1 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:52.294327 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 174/344] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:52.295556 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182baea-d081-4e70-986b-eed90fc8948e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:52.300113 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64401193-7910-4881-9fe8-1d438c396f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:52.300773 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64401193-7910-4881-9fe8-1d438c396f98 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:52.301060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64401193-7910-4881-9fe8-1d438c396f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:52.301332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64401193-7910-4881-9fe8-1d438c396f98 None admin] PATH_INFO: `/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:52.302528 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64401193-7910-4881-9fe8-1d438c396f98 None admin] RBAC: Authorizing `identity:get_project(project_id=076f7405c1874c1ea0d0f44c11ae4d38)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:52.304007 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64401193-7910-4881-9fe8-1d438c396f98 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:52.305974 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 171/345] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:52 2022] GET /identity/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38 => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:52.311524 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64401193-7910-4881-9fe8-1d438c396f98 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:52.387449 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:52.413164 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182baea-d081-4e70-986b-eed90fc8948e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:52.451261 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64401193-7910-4881-9fe8-1d438c396f98 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:52.452260 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a732c4-8eb1-46f4-a2c6-7c7b44fb4681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:52.453227 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a732c4-8eb1-46f4-a2c6-7c7b44fb4681 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:52.453776 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a732c4-8eb1-46f4-a2c6-7c7b44fb4681 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:52.454253 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a732c4-8eb1-46f4-a2c6-7c7b44fb4681 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:52.455169 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a732c4-8eb1-46f4-a2c6-7c7b44fb4681 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:52.455714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a732c4-8eb1-46f4-a2c6-7c7b44fb4681 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:52.457304 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a732c4-8eb1-46f4-a2c6-7c7b44fb4681 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:52.466388 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 175/346] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:29:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:52.467487 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a732c4-8eb1-46f4-a2c6-7c7b44fb4681 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:52.508119 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c707e22-836b-4697-8642-fc94e344bb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:52.508778 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c707e22-836b-4697-8642-fc94e344bb66 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:52.509061 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c707e22-836b-4697-8642-fc94e344bb66 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:52.509344 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c707e22-836b-4697-8642-fc94e344bb66 None admin] PATH_INFO: `/v3/groups/5c4345316deb4e418266f43290528567` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:52.520582 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c707e22-836b-4697-8642-fc94e344bb66 None admin] RBAC: Authorizing `identity:get_group(group_id=5c4345316deb4e418266f43290528567)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:52.522068 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c707e22-836b-4697-8642-fc94e344bb66 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:52.523754 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 172/347] 10.222.0.97 () {56 vars in 1180 bytes} [Tue Jul 26 14:29:52 2022] GET /identity/v3/groups/5c4345316deb4e418266f43290528567 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:52.524681 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c707e22-836b-4697-8642-fc94e344bb66 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:52.535525 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:52.592305 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a732c4-8eb1-46f4-a2c6-7c7b44fb4681 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:52.631135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c244d-882d-431e-9195-13c2563baa11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:52.632378 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c244d-882d-431e-9195-13c2563baa11 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:52.632884 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c244d-882d-431e-9195-13c2563baa11 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:52.633360 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c244d-882d-431e-9195-13c2563baa11 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:52.635090 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c707e22-836b-4697-8642-fc94e344bb66 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:52.642187 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:52.646955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c244d-882d-431e-9195-13c2563baa11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:52.653272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-238c244d-882d-431e-9195-13c2563baa11 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:52.653272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:52.653272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:52.653272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:52.653272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:52.653272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:52.653272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:52.653272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:52.653272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:52.653272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:52.653272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:52.653272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:52.653272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:52.653272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:52.653272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:52.653272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:52.654224 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:52.654224 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:52.654224 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:52.654224 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:52.654224 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:52.654224 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:52.654224 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:52.654224 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:52.654224 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:52.654224 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:52.654224 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:52.654224 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:52.654224 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:52.654224 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:52.654224 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:52.654224 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:52.655015 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:52.655015 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:52.655015 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:52.655015 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:52.655015 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:52.655015 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:52.655015 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:52.655015 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:52.655015 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:52.655015 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:52.655015 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:52.655015 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:29:52.657751 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 176/348] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:52.659311 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68073b64-a03f-4d51-a856-544cb35b2e38 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:52.659833 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68073b64-a03f-4d51-a856-544cb35b2e38 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:52.660406 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68073b64-a03f-4d51-a856-544cb35b2e38 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:52.662023 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 177/349] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:52.662925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68073b64-a03f-4d51-a856-544cb35b2e38 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:52.676857 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461c57b-b30a-4e51-b437-114636ca1da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:52.677449 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461c57b-b30a-4e51-b437-114636ca1da2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:52.677729 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461c57b-b30a-4e51-b437-114636ca1da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:52.678003 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461c57b-b30a-4e51-b437-114636ca1da2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:52.749118 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0461c57b-b30a-4e51-b437-114636ca1da2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:52.750579 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0461c57b-b30a-4e51-b437-114636ca1da2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:52.758584 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:52.780842 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68073b64-a03f-4d51-a856-544cb35b2e38 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:52.820520 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8e46b-252d-4671-9f06-39472d6f031e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:52.821501 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8e46b-252d-4671-9f06-39472d6f031e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:52.821989 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8e46b-252d-4671-9f06-39472d6f031e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:52.822498 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8e46b-252d-4671-9f06-39472d6f031e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:52.823676 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e8e46b-252d-4671-9f06-39472d6f031e None admin] RBAC: Adding query filter params (group.id=5c4345316deb4e418266f43290528567, role.id=e033961a9363456a8510c34414dd839e, scope.project.id=076f7405c1874c1ea0d0f44c11ae4d38) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:52.824274 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e8e46b-252d-4671-9f06-39472d6f031e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:52.825937 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e8e46b-252d-4671-9f06-39472d6f031e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:52.833017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 178/350] 10.222.0.97 () {56 vars in 1376 bytes} [Tue Jul 26 14:29:52 2022] GET /identity/v3/role_assignments?group.id=5c4345316deb4e418266f43290528567&scope.project.id=076f7405c1874c1ea0d0f44c11ae4d38&role.id=e033961a9363456a8510c34414dd839e => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:52.834115 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8e46b-252d-4671-9f06-39472d6f031e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:52.857598 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 173/351] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:29:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:29:52.859068 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb12b84-5a7c-40a3-8e5f-ef84adc241a3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:52.859356 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb12b84-5a7c-40a3-8e5f-ef84adc241a3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:52.859652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb12b84-5a7c-40a3-8e5f-ef84adc241a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:52.878449 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-bcb12b84-5a7c-40a3-8e5f-ef84adc241a3 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:52.879656 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-bcb12b84-5a7c-40a3-8e5f-ef84adc241a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:52.935459 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:52.960649 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 174/352] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:52.961608 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb12b84-5a7c-40a3-8e5f-ef84adc241a3 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:53.001886 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8e46b-252d-4671-9f06-39472d6f031e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:53.015796 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:53.069688 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54dd831-25f0-4a4b-8292-81be91d7fb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:53.071041 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54dd831-25f0-4a4b-8292-81be91d7fb16 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:53.071568 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54dd831-25f0-4a4b-8292-81be91d7fb16 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:53.072229 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54dd831-25f0-4a4b-8292-81be91d7fb16 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:53.073257 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54dd831-25f0-4a4b-8292-81be91d7fb16 None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:53.073852 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54dd831-25f0-4a4b-8292-81be91d7fb16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:53.075585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54dd831-25f0-4a4b-8292-81be91d7fb16 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:53.090071 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 179/353] 10.222.0.97 () {56 vars in 1101 bytes} [Tue Jul 26 14:29:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:53.091347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54dd831-25f0-4a4b-8292-81be91d7fb16 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:53.091925 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb12b84-5a7c-40a3-8e5f-ef84adc241a3 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:53.135064 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72dd82-8143-4d4c-a800-fe90ca74a72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:53.135985 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72dd82-8143-4d4c-a800-fe90ca74a72c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:53.143533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72dd82-8143-4d4c-a800-fe90ca74a72c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:53.143533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72dd82-8143-4d4c-a800-fe90ca74a72c None admin] PATH_INFO: `/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:53.144761 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a72dd82-8143-4d4c-a800-fe90ca74a72c None admin] RBAC: Authorizing `identity:get_project(project_id=076f7405c1874c1ea0d0f44c11ae4d38)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:53.146410 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a72dd82-8143-4d4c-a800-fe90ca74a72c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:53.148346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 175/354] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:52 2022] GET /identity/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:53.149241 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72dd82-8143-4d4c-a800-fe90ca74a72c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:53.224936 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54dd831-25f0-4a4b-8292-81be91d7fb16 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:53.254918 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:53.266199 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745eb32-c205-4682-b608-c773276e4134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:53.267302 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745eb32-c205-4682-b608-c773276e4134 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:53.270344 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745eb32-c205-4682-b608-c773276e4134 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:53.270344 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745eb32-c205-4682-b608-c773276e4134 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:53.274779 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72dd82-8143-4d4c-a800-fe90ca74a72c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:53.277717 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:53.284119 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5745eb32-c205-4682-b608-c773276e4134 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:53.290871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-5745eb32-c205-4682-b608-c773276e4134 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:29:53.290871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:53.290871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:53.290871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:53.290871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:53.290871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:53.290871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:53.290871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:53.290871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:53.290871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:53.290871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:53.290871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:53.290871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:53.290871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:53.290871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:53.290871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:53.291894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:53.291894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:53.291894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:53.291894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:53.291894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:53.291894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:53.291894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:53.291894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:53.291894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:53.291894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:53.291894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:53.291894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:53.291894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:53.291894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:53.291894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:53.291894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:53.292767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:53.292767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:53.292767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:53.292767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:53.292767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:53.292767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:53.292767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:53.292767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:53.292767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:53.292767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:53.292767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:29:53.292767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:29:53.295306 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 180/355] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:53.296441 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745eb32-c205-4682-b608-c773276e4134 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:53.331693 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684356d-41fe-4c1e-8e8f-02061b33ed1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:53.332875 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684356d-41fe-4c1e-8e8f-02061b33ed1e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:53.333190 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684356d-41fe-4c1e-8e8f-02061b33ed1e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:53.333468 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684356d-41fe-4c1e-8e8f-02061b33ed1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:53.424524 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3684356d-41fe-4c1e-8e8f-02061b33ed1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:53.426010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3684356d-41fe-4c1e-8e8f-02061b33ed1e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:53.441703 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745eb32-c205-4682-b608-c773276e4134 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:53.490398 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f91a6d-fae7-4051-9130-f2506dd8eb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:53.491366 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f91a6d-fae7-4051-9130-f2506dd8eb91 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:53.491680 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f91a6d-fae7-4051-9130-f2506dd8eb91 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:53.491965 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f91a6d-fae7-4051-9130-f2506dd8eb91 None admin] PATH_INFO: `/v3/groups/5c4345316deb4e418266f43290528567` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:53.493151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f91a6d-fae7-4051-9130-f2506dd8eb91 None admin] RBAC: Authorizing `identity:get_group(group_id=5c4345316deb4e418266f43290528567)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:53.495251 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f91a6d-fae7-4051-9130-f2506dd8eb91 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:53.497634 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 181/356] 10.222.0.97 () {56 vars in 1180 bytes} [Tue Jul 26 14:29:53 2022] GET /identity/v3/groups/5c4345316deb4e418266f43290528567 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:53.498643 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f307a7-0877-4766-8df0-c24c0fd463d4 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:53.498934 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f307a7-0877-4766-8df0-c24c0fd463d4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:53.499197 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f307a7-0877-4766-8df0-c24c0fd463d4 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:53.500518 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 182/357] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:53.501718 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98012241-bbd1-4c2c-a7ad-cdba4e060274 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:53.501901 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98012241-bbd1-4c2c-a7ad-cdba4e060274 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:53.502167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98012241-bbd1-4c2c-a7ad-cdba4e060274 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:53.503375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 183/358] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:53.504085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98012241-bbd1-4c2c-a7ad-cdba4e060274 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:53.551022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 176/359] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:29:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:29:53.552386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5366372-5e80-4a5a-9943-9d1916a0ea11 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:53.552678 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5366372-5e80-4a5a-9943-9d1916a0ea11 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:53.552954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5366372-5e80-4a5a-9943-9d1916a0ea11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:53.561156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:53.572782 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a5366372-5e80-4a5a-9943-9d1916a0ea11 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:53.574182 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a5366372-5e80-4a5a-9943-9d1916a0ea11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:53.584551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:53.649686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98012241-bbd1-4c2c-a7ad-cdba4e060274 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:53.659033 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 177/360] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:53.659504 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5366372-5e80-4a5a-9943-9d1916a0ea11 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:53.692034 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6fc0ed-9f04-49c4-8816-0c57987dd250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:53.693245 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6fc0ed-9f04-49c4-8816-0c57987dd250 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:53.693713 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6fc0ed-9f04-49c4-8816-0c57987dd250 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:53.694168 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6fc0ed-9f04-49c4-8816-0c57987dd250 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:53.695241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6fc0ed-9f04-49c4-8816-0c57987dd250 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:53.696613 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6fc0ed-9f04-49c4-8816-0c57987dd250 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:53.698148 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6fc0ed-9f04-49c4-8816-0c57987dd250 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:53.708279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 184/361] 10.222.0.97 () {56 vars in 1101 bytes} [Tue Jul 26 14:29:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:53.709782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67718b-8a71-49ff-a3be-0a45bf113748 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:53.710239 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67718b-8a71-49ff-a3be-0a45bf113748 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:53.710666 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67718b-8a71-49ff-a3be-0a45bf113748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:53.731702 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9d67718b-8a71-49ff-a3be-0a45bf113748 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:53.733160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9d67718b-8a71-49ff-a3be-0a45bf113748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:53.788367 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5366372-5e80-4a5a-9943-9d1916a0ea11 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:53.827355 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 185/362] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:29:53.828244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67718b-8a71-49ff-a3be-0a45bf113748 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:53.835621 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ba59c-3744-480d-a340-b3b33bf43416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:53.836256 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ba59c-3744-480d-a340-b3b33bf43416 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:53.836540 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ba59c-3744-480d-a340-b3b33bf43416 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:53.836815 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ba59c-3744-480d-a340-b3b33bf43416 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:53.837617 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ba59c-3744-480d-a340-b3b33bf43416 None admin] RBAC: Adding query filter params (group.id=5c4345316deb4e418266f43290528567, role.id=0602cf371b0f41b390e8a4499ae6bb3c, scope.project.id=076f7405c1874c1ea0d0f44c11ae4d38) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:53.837929 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ba59c-3744-480d-a340-b3b33bf43416 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:53.839436 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ba59c-3744-480d-a340-b3b33bf43416 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:53.841469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:53.847130 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 178/363] 10.222.0.97 () {56 vars in 1376 bytes} [Tue Jul 26 14:29:53 2022] GET /identity/v3/role_assignments?group.id=5c4345316deb4e418266f43290528567&scope.project.id=076f7405c1874c1ea0d0f44c11ae4d38&role.id=0602cf371b0f41b390e8a4499ae6bb3c => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:53.854170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ba59c-3744-480d-a340-b3b33bf43416 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:53.895781 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:53.971149 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67718b-8a71-49ff-a3be-0a45bf113748 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:53.992465 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ba59c-3744-480d-a340-b3b33bf43416 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:54.013858 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa7a6a-0fa4-4a44-87e8-79780943643e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:54.014852 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa7a6a-0fa4-4a44-87e8-79780943643e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:54.016661 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa7a6a-0fa4-4a44-87e8-79780943643e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:54.017155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa7a6a-0fa4-4a44-87e8-79780943643e None admin] PATH_INFO: `/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:54.018856 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaa7a6a-0fa4-4a44-87e8-79780943643e None admin] RBAC: Authorizing `identity:get_project(project_id=5dbf1af61d63414cbf4c33bdc412a9d6)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:54.020591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaa7a6a-0fa4-4a44-87e8-79780943643e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:54.022585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 186/364] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:53 2022] GET /identity/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:54.027685 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa7a6a-0fa4-4a44-87e8-79780943643e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:54.038961 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382ff9c-2da8-4737-887a-be2ea345807c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:54.043466 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382ff9c-2da8-4737-887a-be2ea345807c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:54.043466 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382ff9c-2da8-4737-887a-be2ea345807c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:54.043466 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382ff9c-2da8-4737-887a-be2ea345807c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:54.056161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d382ff9c-2da8-4737-887a-be2ea345807c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:54.060666 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-d382ff9c-2da8-4737-887a-be2ea345807c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:29:54.060666 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:54.060666 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:54.060666 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:54.060666 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:54.060666 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:54.060666 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:54.060666 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:54.060666 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:54.060666 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:54.060666 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:54.060666 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:54.060666 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:54.060666 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:54.060666 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:54.060666 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:54.061548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:54.061548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:54.061548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:54.061548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:54.061548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:54.061548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:54.061548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:54.061548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:54.061548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:54.061548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:54.061548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:54.061548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:54.061548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:54.061548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:54.061548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:54.061548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:54.062346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:54.062346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:54.062346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:54.062346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:54.062346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:54.062346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:54.062346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:54.062346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:54.062346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:54.062346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:54.062346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:29:54.062346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:29:54.071645 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 179/365] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:29:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:54.071645 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382ff9c-2da8-4737-887a-be2ea345807c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:54.152400 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:54.157136 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa7a6a-0fa4-4a44-87e8-79780943643e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:54.172567 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:54.187749 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382ff9c-2da8-4737-887a-be2ea345807c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:54.195048 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace4c5ab-6695-4e37-aa1f-7970b995db0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:54.195882 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4c5ab-6695-4e37-aa1f-7970b995db0c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:54.196259 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4c5ab-6695-4e37-aa1f-7970b995db0c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:54.196653 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4c5ab-6695-4e37-aa1f-7970b995db0c None admin] PATH_INFO: `/v3/users/5a0514276028451a8ad5e80aba582f37` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:54.197939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace4c5ab-6695-4e37-aa1f-7970b995db0c None admin] RBAC: Authorizing `identity:get_user(user_id=5a0514276028451a8ad5e80aba582f37)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:54.199591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace4c5ab-6695-4e37-aa1f-7970b995db0c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:54.201555 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 187/366] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:29:54 2022] GET /identity/v3/users/5a0514276028451a8ad5e80aba582f37 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:54.202410 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace4c5ab-6695-4e37-aa1f-7970b995db0c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:54.231888 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eba2db-35f4-4cc7-9dd6-ae93f47fcd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:54.232574 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eba2db-35f4-4cc7-9dd6-ae93f47fcd4c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:54.232862 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eba2db-35f4-4cc7-9dd6-ae93f47fcd4c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:54.233146 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eba2db-35f4-4cc7-9dd6-ae93f47fcd4c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:54.246972 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eba2db-35f4-4cc7-9dd6-ae93f47fcd4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:54.252317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-50eba2db-35f4-4cc7-9dd6-ae93f47fcd4c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:29:54.252317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:54.252317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:54.252317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:54.252317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:54.252317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:54.252317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:54.252317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:54.252317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:54.252317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:54.252317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:54.252317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:54.252317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:54.252317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:54.252317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:54.252317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:54.253176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:54.253176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:54.253176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:54.253176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:54.253176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:54.253176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:54.253176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:54.253176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:54.253176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:54.253176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:54.253176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:54.253176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:54.253176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:54.253176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:54.253176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:54.253176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:54.253945 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:54.253945 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:54.253945 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:54.253945 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:54.253945 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:54.253945 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:54.253945 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:54.253945 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:54.253945 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:54.253945 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:54.253945 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:29:54.253945 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:29:54.253945 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 180/367] 10.222.0.97 () {56 vars in 1114 bytes} [Tue Jul 26 14:29:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:54.255065 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eba2db-35f4-4cc7-9dd6-ae93f47fcd4c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:54.316531 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace4c5ab-6695-4e37-aa1f-7970b995db0c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:54.353857 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a655f4-80c0-4acf-ae1c-bcffef6a63c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:54.354529 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a655f4-80c0-4acf-ae1c-bcffef6a63c3 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:54.354812 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a655f4-80c0-4acf-ae1c-bcffef6a63c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:54.355197 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a655f4-80c0-4acf-ae1c-bcffef6a63c3 None admin] PATH_INFO: `/v3/groups/5c4345316deb4e418266f43290528567` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:54.356556 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a655f4-80c0-4acf-ae1c-bcffef6a63c3 None admin] RBAC: Authorizing `identity:get_group(group_id=5c4345316deb4e418266f43290528567)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:54.358063 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a655f4-80c0-4acf-ae1c-bcffef6a63c3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:54.360694 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 188/368] 10.222.0.97 () {56 vars in 1180 bytes} [Tue Jul 26 14:29:54 2022] GET /identity/v3/groups/5c4345316deb4e418266f43290528567 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:54.361338 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a655f4-80c0-4acf-ae1c-bcffef6a63c3 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:54.367806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eba2db-35f4-4cc7-9dd6-ae93f47fcd4c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:54.391816 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:54.408738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c4c5a-97ba-4baa-9039-1b910d87ba41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:54.409399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c4c5a-97ba-4baa-9039-1b910d87ba41 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:54.409683 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c4c5a-97ba-4baa-9039-1b910d87ba41 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:54.409962 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c4c5a-97ba-4baa-9039-1b910d87ba41 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:54.410690 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c4c5a-97ba-4baa-9039-1b910d87ba41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:54.410995 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c4c5a-97ba-4baa-9039-1b910d87ba41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:54.412441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c4c5a-97ba-4baa-9039-1b910d87ba41 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:54.420023 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:54.425765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 181/369] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:54.426683 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c4c5a-97ba-4baa-9039-1b910d87ba41 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:54.475286 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a655f4-80c0-4acf-ae1c-bcffef6a63c3 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:54.515610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174e1d7-e452-4949-a913-05bd6beef5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:54.516642 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174e1d7-e452-4949-a913-05bd6beef5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:54.517121 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174e1d7-e452-4949-a913-05bd6beef5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:54.517871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174e1d7-e452-4949-a913-05bd6beef5f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:54.519494 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5174e1d7-e452-4949-a913-05bd6beef5f1 None admin] RBAC: Adding query filter params (role.id=2b922104c92a4cbc9432fbce1c9bda1e, user.id=5a0514276028451a8ad5e80aba582f37, scope.system=all) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:54.520017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5174e1d7-e452-4949-a913-05bd6beef5f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:54.522499 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5174e1d7-e452-4949-a913-05bd6beef5f1 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:54.536775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 189/370] 10.222.0.97 () {56 vars in 1308 bytes} [Tue Jul 26 14:29:54 2022] GET /identity/v3/role_assignments?user.id=5a0514276028451a8ad5e80aba582f37&scope.system=all&role.id=2b922104c92a4cbc9432fbce1c9bda1e => generated 510 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:54.537881 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174e1d7-e452-4949-a913-05bd6beef5f1 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:54.574648 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c4c5a-97ba-4baa-9039-1b910d87ba41 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:54.615219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd7d09-d953-4e59-9334-9d69c43ed711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:54.616273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd7d09-d953-4e59-9334-9d69c43ed711 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:54.616795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd7d09-d953-4e59-9334-9d69c43ed711 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:54.617630 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd7d09-d953-4e59-9334-9d69c43ed711 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:54.619347 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd7d09-d953-4e59-9334-9d69c43ed711 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:54.619978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd7d09-d953-4e59-9334-9d69c43ed711 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:54.621651 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd7d09-d953-4e59-9334-9d69c43ed711 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:54.632856 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 182/371] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:29:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:54.634128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd7d09-d953-4e59-9334-9d69c43ed711 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:54.690324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:54.783003 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd7d09-d953-4e59-9334-9d69c43ed711 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:54.802396 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174e1d7-e452-4949-a913-05bd6beef5f1 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:54.816421 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:54.852100 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7edeae-9043-41cf-aec7-5d119877a625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:54.853075 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400512d-995b-42b5-a5cc-4fa5b1f00f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:54.853375 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7edeae-9043-41cf-aec7-5d119877a625 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:54.853701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400512d-995b-42b5-a5cc-4fa5b1f00f91 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:54.853989 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400512d-995b-42b5-a5cc-4fa5b1f00f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:54.854274 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400512d-995b-42b5-a5cc-4fa5b1f00f91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:54.854544 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7edeae-9043-41cf-aec7-5d119877a625 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:54.855070 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7edeae-9043-41cf-aec7-5d119877a625 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:54.855181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0400512d-995b-42b5-a5cc-4fa5b1f00f91 None admin] RBAC: Adding query filter params (group.id=5c4345316deb4e418266f43290528567, role.id=e033961a9363456a8510c34414dd839e, scope.project.id=5dbf1af61d63414cbf4c33bdc412a9d6) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:54.855421 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0400512d-995b-42b5-a5cc-4fa5b1f00f91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:54.856879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0400512d-995b-42b5-a5cc-4fa5b1f00f91 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:54.865599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 190/372] 10.222.0.97 () {56 vars in 1376 bytes} [Tue Jul 26 14:29:54 2022] GET /identity/v3/role_assignments?group.id=5c4345316deb4e418266f43290528567&scope.project.id=5dbf1af61d63414cbf4c33bdc412a9d6&role.id=e033961a9363456a8510c34414dd839e => generated 260 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:54.866825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400512d-995b-42b5-a5cc-4fa5b1f00f91 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:54.956427 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7edeae-9043-41cf-aec7-5d119877a625 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:54.957925 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7edeae-9043-41cf-aec7-5d119877a625 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:55.010531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:55.020747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400512d-995b-42b5-a5cc-4fa5b1f00f91 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:55.072452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2a442-4417-4804-83ee-104b9cf42d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:55.073647 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2a442-4417-4804-83ee-104b9cf42d8c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:55.074184 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2a442-4417-4804-83ee-104b9cf42d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:55.074709 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2a442-4417-4804-83ee-104b9cf42d8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:55.080534 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 183/373] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:29:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:29:55.086821 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7edeae-9043-41cf-aec7-5d119877a625 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:55.124746 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:55.158215 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2a442-4417-4804-83ee-104b9cf42d8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:55.160996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2a442-4417-4804-83ee-104b9cf42d8c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:55.202652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7edeae-9043-41cf-aec7-5d119877a625 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:55.267893 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e5f06-b484-45db-bb2a-8875cdaeb9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:55.268754 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e5f06-b484-45db-bb2a-8875cdaeb9fd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:55.269132 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e5f06-b484-45db-bb2a-8875cdaeb9fd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:55.269488 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e5f06-b484-45db-bb2a-8875cdaeb9fd None admin] PATH_INFO: `/v3/projects/011581630bbe4c60a042395d12032595` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:55.270809 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e5f06-b484-45db-bb2a-8875cdaeb9fd None admin] RBAC: Authorizing `identity:get_project(project_id=011581630bbe4c60a042395d12032595)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:55.272420 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e5f06-b484-45db-bb2a-8875cdaeb9fd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:55.274510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 184/374] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:55 2022] GET /identity/v3/projects/011581630bbe4c60a042395d12032595 => generated 344 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:55.277960 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 191/375] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:29:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:29:55.280036 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e5f06-b484-45db-bb2a-8875cdaeb9fd None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:55.283880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2a442-4417-4804-83ee-104b9cf42d8c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:55.300187 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:55.363766 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:55.398565 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2a442-4417-4804-83ee-104b9cf42d8c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:55.442758 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6e03b-53e5-4e38-bfca-b6ba01ca13fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:55.443803 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6e03b-53e5-4e38-bfca-b6ba01ca13fe None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:55.444284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6e03b-53e5-4e38-bfca-b6ba01ca13fe None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:55.444750 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6e03b-53e5-4e38-bfca-b6ba01ca13fe None admin] PATH_INFO: `/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:55.446144 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e6e03b-53e5-4e38-bfca-b6ba01ca13fe None admin] RBAC: Authorizing `identity:get_project(project_id=5dbf1af61d63414cbf4c33bdc412a9d6)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:55.446666 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e5f06-b484-45db-bb2a-8875cdaeb9fd None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:55.448100 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e6e03b-53e5-4e38-bfca-b6ba01ca13fe None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:55.450761 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 192/376] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:55 2022] GET /identity/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:55.456609 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6e03b-53e5-4e38-bfca-b6ba01ca13fe None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:55.494994 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887736d4-8b69-413c-a0da-b7c1934696c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:55.496213 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887736d4-8b69-413c-a0da-b7c1934696c8 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:55.496501 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887736d4-8b69-413c-a0da-b7c1934696c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:55.496780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887736d4-8b69-413c-a0da-b7c1934696c8 None admin] PATH_INFO: `/v3/groups/71acea0c5f2f489e957b341cd3328328` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:55.503921 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887736d4-8b69-413c-a0da-b7c1934696c8 None admin] RBAC: Authorizing `identity:get_group(group_id=71acea0c5f2f489e957b341cd3328328)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:55.505408 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887736d4-8b69-413c-a0da-b7c1934696c8 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:55.507488 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 185/377] 10.222.0.97 () {56 vars in 1180 bytes} [Tue Jul 26 14:29:55 2022] GET /identity/v3/groups/71acea0c5f2f489e957b341cd3328328 => generated 229 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:55.513361 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887736d4-8b69-413c-a0da-b7c1934696c8 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:55.568344 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6e03b-53e5-4e38-bfca-b6ba01ca13fe None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:55.605052 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401d10b3-9fc5-4f20-905d-65490783957d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:55.606072 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d10b3-9fc5-4f20-905d-65490783957d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:55.606556 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d10b3-9fc5-4f20-905d-65490783957d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:55.607023 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d10b3-9fc5-4f20-905d-65490783957d None admin] PATH_INFO: `/v3/groups/5c4345316deb4e418266f43290528567` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:55.608414 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401d10b3-9fc5-4f20-905d-65490783957d None admin] RBAC: Authorizing `identity:get_group(group_id=5c4345316deb4e418266f43290528567)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:55.610092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401d10b3-9fc5-4f20-905d-65490783957d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:55.615794 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 193/378] 10.222.0.97 () {56 vars in 1180 bytes} [Tue Jul 26 14:29:55 2022] GET /identity/v3/groups/5c4345316deb4e418266f43290528567 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:55.618298 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401d10b3-9fc5-4f20-905d-65490783957d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:55.631760 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:55.639338 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:55.671151 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887736d4-8b69-413c-a0da-b7c1934696c8 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:55.728401 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401d10b3-9fc5-4f20-905d-65490783957d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:55.767818 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189fb73c-89bb-4d39-b388-8f47f49e3a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:55.768443 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189fb73c-89bb-4d39-b388-8f47f49e3a73 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:55.768722 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189fb73c-89bb-4d39-b388-8f47f49e3a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:55.768995 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189fb73c-89bb-4d39-b388-8f47f49e3a73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:55.769785 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189fb73c-89bb-4d39-b388-8f47f49e3a73 None admin] RBAC: Adding query filter params (group.id=5c4345316deb4e418266f43290528567, role.id=0602cf371b0f41b390e8a4499ae6bb3c, scope.project.id=5dbf1af61d63414cbf4c33bdc412a9d6) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:55.770091 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189fb73c-89bb-4d39-b388-8f47f49e3a73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:55.771537 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189fb73c-89bb-4d39-b388-8f47f49e3a73 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:55.778039 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 194/379] 10.222.0.97 () {56 vars in 1376 bytes} [Tue Jul 26 14:29:55 2022] GET /identity/v3/role_assignments?group.id=5c4345316deb4e418266f43290528567&scope.project.id=5dbf1af61d63414cbf4c33bdc412a9d6&role.id=0602cf371b0f41b390e8a4499ae6bb3c => generated 260 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:55.779024 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c94cf-91e5-44d7-9b99-cdfe367a2893 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:55.779306 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c94cf-91e5-44d7-9b99-cdfe367a2893 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:55.779602 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c94cf-91e5-44d7-9b99-cdfe367a2893 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:55.780128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732d588-0815-4af7-a7d1-e9bb62d6c701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:55.780776 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732d588-0815-4af7-a7d1-e9bb62d6c701 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:55.781065 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732d588-0815-4af7-a7d1-e9bb62d6c701 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:55.781215 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 195/380] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:55.781383 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732d588-0815-4af7-a7d1-e9bb62d6c701 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:55.782198 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a732d588-0815-4af7-a7d1-e9bb62d6c701 None admin] RBAC: Adding query filter params (group.id=71acea0c5f2f489e957b341cd3328328, role.id=51b2d10dfffb4b75b186f3d8b15e2a78, scope.project.id=011581630bbe4c60a042395d12032595) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:55.782505 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a732d588-0815-4af7-a7d1-e9bb62d6c701 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:55.784037 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a732d588-0815-4af7-a7d1-e9bb62d6c701 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:55.788849 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc60cd-0aa2-4c4e-815a-6a9b2f2c0f30 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:55.789289 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc60cd-0aa2-4c4e-815a-6a9b2f2c0f30 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:55.789576 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc60cd-0aa2-4c4e-815a-6a9b2f2c0f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:55.793793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 186/381] 10.222.0.97 () {56 vars in 1376 bytes} [Tue Jul 26 14:29:55 2022] GET /identity/v3/role_assignments?group.id=71acea0c5f2f489e957b341cd3328328&scope.project.id=011581630bbe4c60a042395d12032595&role.id=51b2d10dfffb4b75b186f3d8b15e2a78 => generated 260 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:55.819105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-45bc60cd-0aa2-4c4e-815a-6a9b2f2c0f30 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:55.820365 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-45bc60cd-0aa2-4c4e-815a-6a9b2f2c0f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:55.937283 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 196/382] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:29:55.977332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732d588-0815-4af7-a7d1-e9bb62d6c701 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:56.107847 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732d588-0815-4af7-a7d1-e9bb62d6c701 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:56.151020 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2521c5-0ea9-47de-be22-939639dbc292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:56.151772 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2521c5-0ea9-47de-be22-939639dbc292 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:56.152057 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2521c5-0ea9-47de-be22-939639dbc292 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:56.152336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2521c5-0ea9-47de-be22-939639dbc292 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:56.164975 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:56.170200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2521c5-0ea9-47de-be22-939639dbc292 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:56.175168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-cf2521c5-0ea9-47de-be22-939639dbc292 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:56.175168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:56.175168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:56.175168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:56.175168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:56.175168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:56.175168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:56.175168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:56.175168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:56.175168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:56.175168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:56.175168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:56.175168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:56.175168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:56.175168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:56.175168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:56.176017 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:56.176017 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:56.176017 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:56.176017 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:56.176017 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:56.176017 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:56.176017 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:56.176017 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:56.176017 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:56.176017 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:56.176017 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:56.176017 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:56.176017 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:56.176017 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:56.176017 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:56.176017 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:56.176788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:56.176788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:56.176788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:56.176788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:56.176788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:56.176788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:56.176788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:56.176788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:56.176788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:56.176788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:56.176788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:56.176788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:29:56.176788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 187/383] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:56.182266 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc60cd-0aa2-4c4e-815a-6a9b2f2c0f30 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:56.193004 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:56.301282 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc60cd-0aa2-4c4e-815a-6a9b2f2c0f30 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:56.352135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8552393-57ea-4f52-bc71-61881c6d6ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:56.352776 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8552393-57ea-4f52-bc71-61881c6d6ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:56.353062 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8552393-57ea-4f52-bc71-61881c6d6ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:56.353341 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8552393-57ea-4f52-bc71-61881c6d6ff4 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:56.354073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8552393-57ea-4f52-bc71-61881c6d6ff4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:56.354382 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8552393-57ea-4f52-bc71-61881c6d6ff4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:56.355889 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8552393-57ea-4f52-bc71-61881c6d6ff4 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:56.364939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 197/384] 10.222.0.97 () {56 vars in 1101 bytes} [Tue Jul 26 14:29:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:56.371120 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2521c5-0ea9-47de-be22-939639dbc292 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:56.496254 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2521c5-0ea9-47de-be22-939639dbc292 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:56.531608 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d4b1f-20c0-4618-bfce-8a3256b9eefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:56.532642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d4b1f-20c0-4618-bfce-8a3256b9eefa None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:56.533119 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d4b1f-20c0-4618-bfce-8a3256b9eefa None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:56.533588 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d4b1f-20c0-4618-bfce-8a3256b9eefa None admin] PATH_INFO: `/v3/groups/5c4345316deb4e418266f43290528567` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:56.534932 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d4b1f-20c0-4618-bfce-8a3256b9eefa None admin] RBAC: Authorizing `identity:get_group(group_id=5c4345316deb4e418266f43290528567)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:56.536620 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d4b1f-20c0-4618-bfce-8a3256b9eefa None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:56.538406 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 188/385] 10.222.0.97 () {56 vars in 1180 bytes} [Tue Jul 26 14:29:56 2022] GET /identity/v3/groups/5c4345316deb4e418266f43290528567 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:56.544474 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8552393-57ea-4f52-bc71-61881c6d6ff4 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:56.572960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4faaf-991e-42f1-ab2b-29c04b099358 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:56.573757 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4faaf-991e-42f1-ab2b-29c04b099358 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:56.574562 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4faaf-991e-42f1-ab2b-29c04b099358 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:56.576151 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 189/386] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:29:56.584548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f754c-c934-4748-a917-01c266bc252f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:56.585074 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f754c-c934-4748-a917-01c266bc252f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:56.585544 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f754c-c934-4748-a917-01c266bc252f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:56.606948 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f34f754c-c934-4748-a917-01c266bc252f None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:56.612858 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f34f754c-c934-4748-a917-01c266bc252f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:56.618866 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:56.682549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8552393-57ea-4f52-bc71-61881c6d6ff4 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:56.685256 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:56.693738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 190/387] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:29:56.732191 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f754c-c934-4748-a917-01c266bc252f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:56.732790 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634ca7b-00d8-43e5-9afb-a8bf066dea42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:56.733458 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634ca7b-00d8-43e5-9afb-a8bf066dea42 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:56.733739 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634ca7b-00d8-43e5-9afb-a8bf066dea42 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:56.734016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634ca7b-00d8-43e5-9afb-a8bf066dea42 None admin] PATH_INFO: `/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:56.736087 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5634ca7b-00d8-43e5-9afb-a8bf066dea42 None admin] RBAC: Authorizing `identity:get_project(project_id=076f7405c1874c1ea0d0f44c11ae4d38)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:56.737646 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5634ca7b-00d8-43e5-9afb-a8bf066dea42 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:56.742552 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 198/388] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:56 2022] GET /identity/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:56.745016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634ca7b-00d8-43e5-9afb-a8bf066dea42 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:56.867355 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634ca7b-00d8-43e5-9afb-a8bf066dea42 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:56.870370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f754c-c934-4748-a917-01c266bc252f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:56.901905 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:56.910933 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f8afe-1fb5-459f-ba2e-ce4e84b8360a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:56.911648 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f8afe-1fb5-459f-ba2e-ce4e84b8360a None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:56.911981 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f8afe-1fb5-459f-ba2e-ce4e84b8360a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:56.912211 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f8afe-1fb5-459f-ba2e-ce4e84b8360a None admin] PATH_INFO: `/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38/groups/5c4345316deb4e418266f43290528567/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:56.916944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1f8afe-1fb5-459f-ba2e-ce4e84b8360a None admin] RBAC: Authorizing `identity:create_grant(project_id=076f7405c1874c1ea0d0f44c11ae4d38, group_id=5c4345316deb4e418266f43290528567, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:56.918235 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1f8afe-1fb5-459f-ba2e-ce4e84b8360a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:56.934230 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8da8ba-2fc2-4768-939e-b14400b4048a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:56.935561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8da8ba-2fc2-4768-939e-b14400b4048a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:56.935561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8da8ba-2fc2-4768-939e-b14400b4048a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:56.935561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8da8ba-2fc2-4768-939e-b14400b4048a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:56.947527 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8da8ba-2fc2-4768-939e-b14400b4048a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:56.952722 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-fb8da8ba-2fc2-4768-939e-b14400b4048a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:29:56.952722 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:56.952722 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:56.952722 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:56.952722 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:56.952722 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:56.952722 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:56.952722 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:56.952722 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:56.952722 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:56.952722 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:56.952722 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:56.952722 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:56.952722 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:56.952722 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:56.952722 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:56.953585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:56.953585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:56.953585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:56.953585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:56.953585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:56.953585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:56.953585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:56.953585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:56.953585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:56.953585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:56.953585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:56.953585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:56.953585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:56.953585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:56.953585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:56.953585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:56.954370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:56.954370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:56.954370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:56.954370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:56.954370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:56.954370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:56.954370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:56.954370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:56.954370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:56.954370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:56.954370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:29:56.954370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:29:56.954370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 191/389] 10.222.0.97 () {56 vars in 1114 bytes} [Tue Jul 26 14:29:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:29:56.959704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8da8ba-2fc2-4768-939e-b14400b4048a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:57.014898 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 199/390] 10.222.0.97 () {58 vars in 1442 bytes} [Tue Jul 26 14:29:56 2022] PUT /identity/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38/groups/5c4345316deb4e418266f43290528567/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:29:57.043646 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:57.102659 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8da8ba-2fc2-4768-939e-b14400b4048a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:57.140751 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a807f-547e-4e00-9b5a-71a665808be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:57.141794 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a807f-547e-4e00-9b5a-71a665808be6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:57.142575 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a807f-547e-4e00-9b5a-71a665808be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:57.143055 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a807f-547e-4e00-9b5a-71a665808be6 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:57.144060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21a807f-547e-4e00-9b5a-71a665808be6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:57.144664 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21a807f-547e-4e00-9b5a-71a665808be6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:57.146284 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21a807f-547e-4e00-9b5a-71a665808be6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:57.157092 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 192/391] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:29:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:57.162447 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f8afe-1fb5-459f-ba2e-ce4e84b8360a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:57.314711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f8afe-1fb5-459f-ba2e-ce4e84b8360a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:57.366971 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e07b4-5d72-4d6e-a0d7-651e1027a426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:57.367667 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e07b4-5d72-4d6e-a0d7-651e1027a426 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:57.367953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e07b4-5d72-4d6e-a0d7-651e1027a426 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:57.368229 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e07b4-5d72-4d6e-a0d7-651e1027a426 None admin] PATH_INFO: `/v3/groups/5c4345316deb4e418266f43290528567` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:57.369413 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09e07b4-5d72-4d6e-a0d7-651e1027a426 None admin] RBAC: Authorizing `identity:get_group(group_id=5c4345316deb4e418266f43290528567)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:57.370899 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09e07b4-5d72-4d6e-a0d7-651e1027a426 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:57.373548 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 200/392] 10.222.0.97 () {56 vars in 1180 bytes} [Tue Jul 26 14:29:57 2022] GET /identity/v3/groups/5c4345316deb4e418266f43290528567 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:57.378125 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a807f-547e-4e00-9b5a-71a665808be6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:57.503506 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a807f-547e-4e00-9b5a-71a665808be6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:57.513092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c8e96-b793-4993-91f1-651c111a6cae None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:57.513385 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c8e96-b793-4993-91f1-651c111a6cae None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:57.513665 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c8e96-b793-4993-91f1-651c111a6cae None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:57.517199 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 201/393] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:57.522895 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b66716-6d17-48ec-93ea-c7526ce6180b None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:57.523185 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b66716-6d17-48ec-93ea-c7526ce6180b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:57.523735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b66716-6d17-48ec-93ea-c7526ce6180b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:57.524992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:57.554545 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa2b4e-b686-4e0f-bb15-13a8f9ab4fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:57.555205 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa2b4e-b686-4e0f-bb15-13a8f9ab4fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:57.555604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa2b4e-b686-4e0f-bb15-13a8f9ab4fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:57.555886 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa2b4e-b686-4e0f-bb15-13a8f9ab4fd0 None admin] PATH_INFO: `/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:57.556885 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-60b66716-6d17-48ec-93ea-c7526ce6180b None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:57.557220 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa2b4e-b686-4e0f-bb15-13a8f9ab4fd0 None admin] RBAC: Authorizing `identity:get_project(project_id=076f7405c1874c1ea0d0f44c11ae4d38)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:57.558156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-60b66716-6d17-48ec-93ea-c7526ce6180b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:57.558652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa2b4e-b686-4e0f-bb15-13a8f9ab4fd0 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:57.565202 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 193/394] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:57 2022] GET /identity/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:57.566012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa2b4e-b686-4e0f-bb15-13a8f9ab4fd0 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:57.674271 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa2b4e-b686-4e0f-bb15-13a8f9ab4fd0 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:57.710011 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb29b2d-4568-4871-bd65-62c90bf50b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:57.710692 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb29b2d-4568-4871-bd65-62c90bf50b8d None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:57.710975 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb29b2d-4568-4871-bd65-62c90bf50b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:57.711255 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb29b2d-4568-4871-bd65-62c90bf50b8d None admin] PATH_INFO: `/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38/groups/5c4345316deb4e418266f43290528567/roles/0602cf371b0f41b390e8a4499ae6bb3c` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:57.713621 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb29b2d-4568-4871-bd65-62c90bf50b8d None admin] RBAC: Authorizing `identity:create_grant(project_id=076f7405c1874c1ea0d0f44c11ae4d38, group_id=5c4345316deb4e418266f43290528567, role_id=0602cf371b0f41b390e8a4499ae6bb3c)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:57.715891 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb29b2d-4568-4871-bd65-62c90bf50b8d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:57.721453 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:57.738332 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 202/395] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:29:57.763216 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b66716-6d17-48ec-93ea-c7526ce6180b None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:57.819774 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 194/396] 10.222.0.97 () {58 vars in 1442 bytes} [Tue Jul 26 14:29:57 2022] PUT /identity/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38/groups/5c4345316deb4e418266f43290528567/roles/0602cf371b0f41b390e8a4499ae6bb3c => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:29:57.911726 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b66716-6d17-48ec-93ea-c7526ce6180b None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:57.964652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd354e1-2990-470a-8244-ab1e021a38f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:57.965330 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd354e1-2990-470a-8244-ab1e021a38f3 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:57.965611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd354e1-2990-470a-8244-ab1e021a38f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:57.965881 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd354e1-2990-470a-8244-ab1e021a38f3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:57.984677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd354e1-2990-470a-8244-ab1e021a38f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:57.989853 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-1bd354e1-2990-470a-8244-ab1e021a38f3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:57.989853 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:57.989853 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:57.989853 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:57.989853 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:57.989853 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:57.989853 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:57.989853 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:57.989853 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:57.989853 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:57.989853 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:57.989853 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:57.989853 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:57.989853 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:57.989853 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:57.989853 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:57.990767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:57.990767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:57.990767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:57.990767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:57.990767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:57.990767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:57.990767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:57.990767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:57.990767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:57.990767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:57.990767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:57.990767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:57.990767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:57.990767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:57.990767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:57.990767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:57.991652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:57.991652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:57.991652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:57.991652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:57.991652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:57.991652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:57.991652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:57.991652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:57.991652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:57.991652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:57.991652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:29:57.991652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:29:57.991652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 203/397] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:29:57.998913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb29b2d-4568-4871-bd65-62c90bf50b8d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:58.071099 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:58.135572 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb29b2d-4568-4871-bd65-62c90bf50b8d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:58.177306 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd34c1-ea5f-4d20-9a31-270e8777b310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:58.177939 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd34c1-ea5f-4d20-9a31-270e8777b310 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:58.178321 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd34c1-ea5f-4d20-9a31-270e8777b310 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:58.178617 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd34c1-ea5f-4d20-9a31-270e8777b310 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:58.179347 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd34c1-ea5f-4d20-9a31-270e8777b310 None admin] RBAC: Adding query filter params (name=member) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:58.179689 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd34c1-ea5f-4d20-9a31-270e8777b310 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:58.181215 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd34c1-ea5f-4d20-9a31-270e8777b310 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:58.193122 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 195/398] 10.222.0.97 () {56 vars in 1101 bytes} [Tue Jul 26 14:29:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:58.200719 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd354e1-2990-470a-8244-ab1e021a38f3 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:58.271617 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:58.332276 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd354e1-2990-470a-8244-ab1e021a38f3 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:58.377911 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016823f-6cbe-4698-bffe-8aa83f7f1a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:58.378663 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016823f-6cbe-4698-bffe-8aa83f7f1a27 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:58.378867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016823f-6cbe-4698-bffe-8aa83f7f1a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:58.379139 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016823f-6cbe-4698-bffe-8aa83f7f1a27 None admin] PATH_INFO: `/v3/groups/5c4345316deb4e418266f43290528567` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:58.384220 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4016823f-6cbe-4698-bffe-8aa83f7f1a27 None admin] RBAC: Authorizing `identity:get_group(group_id=5c4345316deb4e418266f43290528567)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:58.385588 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4016823f-6cbe-4698-bffe-8aa83f7f1a27 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:58.388083 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 204/399] 10.222.0.97 () {56 vars in 1180 bytes} [Tue Jul 26 14:29:58 2022] GET /identity/v3/groups/5c4345316deb4e418266f43290528567 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:58.393515 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd34c1-ea5f-4d20-9a31-270e8777b310 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:58.519394 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd34c1-ea5f-4d20-9a31-270e8777b310 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:58.532909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:58.564236 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eee796-af66-4ee2-ab5b-011e04cbcd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:58.564901 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eee796-af66-4ee2-ab5b-011e04cbcd12 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:58.565185 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eee796-af66-4ee2-ab5b-011e04cbcd12 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:58.565463 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eee796-af66-4ee2-ab5b-011e04cbcd12 None admin] PATH_INFO: `/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:58.566726 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eee796-af66-4ee2-ab5b-011e04cbcd12 None admin] RBAC: Authorizing `identity:get_project(project_id=5dbf1af61d63414cbf4c33bdc412a9d6)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:58.568259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eee796-af66-4ee2-ab5b-011e04cbcd12 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:58.570299 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 196/400] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:58 2022] GET /identity/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:58.576119 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016823f-6cbe-4698-bffe-8aa83f7f1a27 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:58.590870 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d0553-7ca9-4bae-9b1b-f61a9bbd6eba None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:58.591139 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d0553-7ca9-4bae-9b1b-f61a9bbd6eba None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:58.591348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d0553-7ca9-4bae-9b1b-f61a9bbd6eba None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:58.593082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 197/401] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:29:58.598007 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b24032-c8b4-4dfd-b9f2-a4b2e89ec4ab None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:58.598288 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b24032-c8b4-4dfd-b9f2-a4b2e89ec4ab None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:58.598567 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b24032-c8b4-4dfd-b9f2-a4b2e89ec4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:58.620027 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-88b24032-c8b4-4dfd-b9f2-a4b2e89ec4ab None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:58.621522 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-88b24032-c8b4-4dfd-b9f2-a4b2e89ec4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:58.707380 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 198/402] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:29:58.725741 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016823f-6cbe-4698-bffe-8aa83f7f1a27 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:58.736225 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b24032-c8b4-4dfd-b9f2-a4b2e89ec4ab None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:58.750140 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:58.770482 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3f52b-aa9f-4a45-b93a-aa703653c043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:58.771641 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3f52b-aa9f-4a45-b93a-aa703653c043 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:58.772129 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3f52b-aa9f-4a45-b93a-aa703653c043 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:58.772623 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3f52b-aa9f-4a45-b93a-aa703653c043 None admin] PATH_INFO: `/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6/groups/5c4345316deb4e418266f43290528567/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:58.775460 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b3f52b-aa9f-4a45-b93a-aa703653c043 None admin] RBAC: Authorizing `identity:create_grant(project_id=5dbf1af61d63414cbf4c33bdc412a9d6, group_id=5c4345316deb4e418266f43290528567, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:58.777239 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b3f52b-aa9f-4a45-b93a-aa703653c043 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:58.858179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b24032-c8b4-4dfd-b9f2-a4b2e89ec4ab None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:58.868574 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:58.898066 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ddf136-a152-4b9c-bae1-b37821e00ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:58.898765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddf136-a152-4b9c-bae1-b37821e00ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:58.899036 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddf136-a152-4b9c-bae1-b37821e00ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:58.899375 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddf136-a152-4b9c-bae1-b37821e00ac6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:58.911040 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ddf136-a152-4b9c-bae1-b37821e00ac6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:58.912927 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 205/403] 10.222.0.97 () {58 vars in 1442 bytes} [Tue Jul 26 14:29:58 2022] PUT /identity/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6/groups/5c4345316deb4e418266f43290528567/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:29:58.914812 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6ccb7-40e3-4b8c-80d3-29e0845bb395 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:58.939948 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6ccb7-40e3-4b8c-80d3-29e0845bb395 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:58.939948 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6ccb7-40e3-4b8c-80d3-29e0845bb395 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:58.940161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-a1ddf136-a152-4b9c-bae1-b37821e00ac6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:29:58.940161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:58.940161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:58.940161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:58.940161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:58.940161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:58.940161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:58.940161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:58.940161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:58.940161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:58.940161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:58.940161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:58.940161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:58.940161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:58.940161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:58.940161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:58.940957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:58.940957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:58.940957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:58.940957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:58.940957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:58.940957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:58.940957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:58.940957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:58.940957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:58.940957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:58.940957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:58.940957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:58.940957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:58.940957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:58.940957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:58.940957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:58.941791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:58.941791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:58.941791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:58.941791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:58.941791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:58.941791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:58.941791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:58.941791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:58.941791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:58.941791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:58.941791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:29:58.941791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:29:58.942377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 206/404] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:58.942444 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 199/405] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:29:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:29:58.942508 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08b3a7-d755-4602-8497-32e996f3b419 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:58.942508 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08b3a7-d755-4602-8497-32e996f3b419 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:58.942634 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ddf136-a152-4b9c-bae1-b37821e00ac6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:58.942698 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08b3a7-d755-4602-8497-32e996f3b419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:58.953476 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-5c08b3a7-d755-4602-8497-32e996f3b419 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:58.955428 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5c08b3a7-d755-4602-8497-32e996f3b419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:29:59.072521 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ddf136-a152-4b9c-bae1-b37821e00ac6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:59.077822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 207/406] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:29:59.105687 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08b3a7-d755-4602-8497-32e996f3b419 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:59.147805 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051a1c3-5c94-4a80-87f2-b946b18cc68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:59.148352 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051a1c3-5c94-4a80-87f2-b946b18cc68d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:59.148631 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051a1c3-5c94-4a80-87f2-b946b18cc68d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:59.148904 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051a1c3-5c94-4a80-87f2-b946b18cc68d None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:59.149631 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6051a1c3-5c94-4a80-87f2-b946b18cc68d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:59.149936 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6051a1c3-5c94-4a80-87f2-b946b18cc68d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:59.151340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6051a1c3-5c94-4a80-87f2-b946b18cc68d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:59.170540 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 200/407] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:29:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:59.177414 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051a1c3-5c94-4a80-87f2-b946b18cc68d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:59.209547 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:59.281073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08b3a7-d755-4602-8497-32e996f3b419 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:59.301222 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:59.323020 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051a1c3-5c94-4a80-87f2-b946b18cc68d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:59.357413 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15edeb79-d49e-4330-a2f6-63bd9d72f6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:59.358069 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15edeb79-d49e-4330-a2f6-63bd9d72f6ee None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:59.358350 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15edeb79-d49e-4330-a2f6-63bd9d72f6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:59.358624 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15edeb79-d49e-4330-a2f6-63bd9d72f6ee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:59.380666 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15edeb79-d49e-4330-a2f6-63bd9d72f6ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:29:59.385672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-15edeb79-d49e-4330-a2f6-63bd9d72f6ee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:29:59.385672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:29:59.385672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:29:59.385672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:29:59.385672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:29:59.385672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:29:59.385672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:29:59.385672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:29:59.385672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:29:59.385672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:29:59.385672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:29:59.385672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:29:59.385672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:29:59.385672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:29:59.385672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:29:59.385672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:29:59.386595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:29:59.386595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:29:59.386595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:29:59.386595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:29:59.386595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:29:59.386595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:29:59.386595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:29:59.386595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:29:59.386595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:29:59.386595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:29:59.386595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:29:59.386595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:29:59.386595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:29:59.386595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:29:59.386595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:29:59.386595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:29:59.387375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:29:59.387375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:29:59.387375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:29:59.387375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:29:59.387375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:29:59.387375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:29:59.387375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:29:59.387375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:29:59.387375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:29:59.387375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:29:59.387375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:29:59.387375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:29:59.388447 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 208/408] 10.222.0.97 () {56 vars in 1114 bytes} [Tue Jul 26 14:29:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:29:59.396639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15edeb79-d49e-4330-a2f6-63bd9d72f6ee None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:59.401943 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b6d63-25c6-4b00-a5d6-eac776959efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:59.402553 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b6d63-25c6-4b00-a5d6-eac776959efb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:59.407423 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b6d63-25c6-4b00-a5d6-eac776959efb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:59.407423 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b6d63-25c6-4b00-a5d6-eac776959efb None admin] PATH_INFO: `/v3/groups/71acea0c5f2f489e957b341cd3328328` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:59.409175 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b6d63-25c6-4b00-a5d6-eac776959efb None admin] RBAC: Authorizing `identity:get_group(group_id=71acea0c5f2f489e957b341cd3328328)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:59.410619 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b6d63-25c6-4b00-a5d6-eac776959efb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:59.412275 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 201/409] 10.222.0.97 () {56 vars in 1180 bytes} [Tue Jul 26 14:29:59 2022] GET /identity/v3/groups/71acea0c5f2f489e957b341cd3328328 => generated 229 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:59.418353 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b6d63-25c6-4b00-a5d6-eac776959efb None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:59.597086 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15edeb79-d49e-4330-a2f6-63bd9d72f6ee None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:59.619232 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:59.634176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b6d63-25c6-4b00-a5d6-eac776959efb None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:59.645284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6004ec8-9f4c-49d2-bf4c-c8db7cb9b3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:59.646111 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6004ec8-9f4c-49d2-bf4c-c8db7cb9b3b3 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:59.646885 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6004ec8-9f4c-49d2-bf4c-c8db7cb9b3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:59.649437 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6004ec8-9f4c-49d2-bf4c-c8db7cb9b3b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:59.650350 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6004ec8-9f4c-49d2-bf4c-c8db7cb9b3b3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:29:59.650807 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6004ec8-9f4c-49d2-bf4c-c8db7cb9b3b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:59.654115 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6004ec8-9f4c-49d2-bf4c-c8db7cb9b3b3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:59.668795 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 209/410] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:29:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:59.680355 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6004ec8-9f4c-49d2-bf4c-c8db7cb9b3b3 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:59.690004 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb4cfc-bfcd-40f4-84a7-c2baf16a936c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:59.697396 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb4cfc-bfcd-40f4-84a7-c2baf16a936c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:59.697396 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb4cfc-bfcd-40f4-84a7-c2baf16a936c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:59.697396 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb4cfc-bfcd-40f4-84a7-c2baf16a936c None admin] PATH_INFO: `/v3/projects/011581630bbe4c60a042395d12032595` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:59.698786 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddb4cfc-bfcd-40f4-84a7-c2baf16a936c None admin] RBAC: Authorizing `identity:get_project(project_id=011581630bbe4c60a042395d12032595)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:59.700964 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddb4cfc-bfcd-40f4-84a7-c2baf16a936c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:59.704959 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 202/411] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:59 2022] GET /identity/v3/projects/011581630bbe4c60a042395d12032595 => generated 344 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:29:59.709937 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb4cfc-bfcd-40f4-84a7-c2baf16a936c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:59.738755 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:29:59.830512 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6004ec8-9f4c-49d2-bf4c-c8db7cb9b3b3 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:59.844430 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb4cfc-bfcd-40f4-84a7-c2baf16a936c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:29:59.868414 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601978ef-14d8-4102-adcb-66fdcbc99a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:59.869133 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601978ef-14d8-4102-adcb-66fdcbc99a31 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:59.869421 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601978ef-14d8-4102-adcb-66fdcbc99a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:59.869698 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601978ef-14d8-4102-adcb-66fdcbc99a31 None admin] PATH_INFO: `/v3/groups/5c4345316deb4e418266f43290528567` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:59.870967 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601978ef-14d8-4102-adcb-66fdcbc99a31 None admin] RBAC: Authorizing `identity:get_group(group_id=5c4345316deb4e418266f43290528567)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:59.872566 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601978ef-14d8-4102-adcb-66fdcbc99a31 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:59.874380 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 210/412] 10.222.0.97 () {56 vars in 1180 bytes} [Tue Jul 26 14:29:59 2022] GET /identity/v3/groups/5c4345316deb4e418266f43290528567 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:29:59.875598 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b410e6a-9ed5-47df-af69-b88e39c8d38a None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:59.875894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b410e6a-9ed5-47df-af69-b88e39c8d38a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:59.876176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b410e6a-9ed5-47df-af69-b88e39c8d38a None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:59.877673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 211/413] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:29:59.879723 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b410e6a-9ed5-47df-af69-b88e39c8d38a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:29:59.884936 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f170a83e-aa32-4b66-851c-304c61f261ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:29:59.885634 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f170a83e-aa32-4b66-851c-304c61f261ab None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:59.886483 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f170a83e-aa32-4b66-851c-304c61f261ab None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:59.886741 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f170a83e-aa32-4b66-851c-304c61f261ab None admin] PATH_INFO: `/v3/projects/011581630bbe4c60a042395d12032595/groups/71acea0c5f2f489e957b341cd3328328/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:59.888766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f170a83e-aa32-4b66-851c-304c61f261ab None admin] RBAC: Authorizing `identity:create_grant(project_id=011581630bbe4c60a042395d12032595, group_id=71acea0c5f2f489e957b341cd3328328, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:29:59.890216 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f170a83e-aa32-4b66-851c-304c61f261ab None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:29:59.970989 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 203/414] 10.222.0.97 () {58 vars in 1442 bytes} [Tue Jul 26 14:29:59 2022] PUT /identity/v3/projects/011581630bbe4c60a042395d12032595/groups/71acea0c5f2f489e957b341cd3328328/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:29:59.973738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad916fe2-8697-4193-bcf3-d9c767e17986 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:29:59.973918 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad916fe2-8697-4193-bcf3-d9c767e17986 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:29:59.974201 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad916fe2-8697-4193-bcf3-d9c767e17986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:29:59.995801 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ad916fe2-8697-4193-bcf3-d9c767e17986 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:29:59.997259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ad916fe2-8697-4193-bcf3-d9c767e17986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:00.009313 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:00.020481 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b410e6a-9ed5-47df-af69-b88e39c8d38a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:00.020955 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:00.099364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd86989-c194-4c59-8f4a-30ff4b0bdd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:00.100065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd86989-c194-4c59-8f4a-30ff4b0bdd60 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:00.100365 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd86989-c194-4c59-8f4a-30ff4b0bdd60 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:00.100631 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd86989-c194-4c59-8f4a-30ff4b0bdd60 None admin] PATH_INFO: `/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:00.102125 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd86989-c194-4c59-8f4a-30ff4b0bdd60 None admin] RBAC: Authorizing `identity:get_project(project_id=5dbf1af61d63414cbf4c33bdc412a9d6)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:00.103813 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd86989-c194-4c59-8f4a-30ff4b0bdd60 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:00.105891 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 212/415] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:29:59 2022] GET /identity/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6 => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:30:00.111383 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd86989-c194-4c59-8f4a-30ff4b0bdd60 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:00.122833 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 204/416] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:29:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:30:00.168771 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad916fe2-8697-4193-bcf3-d9c767e17986 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:00.254116 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd86989-c194-4c59-8f4a-30ff4b0bdd60 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:00.295015 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad916fe2-8697-4193-bcf3-d9c767e17986 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:00.296192 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c84cd3-239b-4073-9e4a-c860cd76694d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:00.296881 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c84cd3-239b-4073-9e4a-c860cd76694d None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:00.297176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c84cd3-239b-4073-9e4a-c860cd76694d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:00.297567 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c84cd3-239b-4073-9e4a-c860cd76694d None admin] PATH_INFO: `/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6/groups/5c4345316deb4e418266f43290528567/roles/0602cf371b0f41b390e8a4499ae6bb3c` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:00.300131 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c84cd3-239b-4073-9e4a-c860cd76694d None admin] RBAC: Authorizing `identity:create_grant(project_id=5dbf1af61d63414cbf4c33bdc412a9d6, group_id=5c4345316deb4e418266f43290528567, role_id=0602cf371b0f41b390e8a4499ae6bb3c)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:00.307427 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c84cd3-239b-4073-9e4a-c860cd76694d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:00.339069 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8858e25-3889-4256-a119-84ddb245b41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:00.339797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8858e25-3889-4256-a119-84ddb245b41a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:00.340084 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8858e25-3889-4256-a119-84ddb245b41a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:00.340358 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8858e25-3889-4256-a119-84ddb245b41a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:00.351217 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8858e25-3889-4256-a119-84ddb245b41a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:00.356365 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-a8858e25-3889-4256-a119-84ddb245b41a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:30:00.356365 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:00.356365 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:00.356365 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:00.356365 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:00.356365 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:00.356365 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:00.356365 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:00.356365 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:00.356365 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:00.356365 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:00.356365 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:00.356365 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:00.356365 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:00.356365 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:00.356365 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:00.357200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:00.357200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:00.357200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:00.357200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:00.357200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:00.357200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:00.357200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:00.357200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:00.357200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:00.357200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:00.357200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:00.357200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:00.357200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:00.357200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:00.357200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:00.357200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:00.358056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:00.358056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:00.358056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:00.358056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:00.358056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:00.358056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:00.358056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:00.358056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:00.358056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:00.358056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:00.358056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:30:00.358056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:00.358056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 205/417] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:30:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:00.363518 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8858e25-3889-4256-a119-84ddb245b41a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:00.379460 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 213/418] 10.222.0.97 () {58 vars in 1442 bytes} [Tue Jul 26 14:30:00 2022] PUT /identity/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6/groups/5c4345316deb4e418266f43290528567/roles/0602cf371b0f41b390e8a4499ae6bb3c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:30:00.388224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:00.485521 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8858e25-3889-4256-a119-84ddb245b41a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:00.535784 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a020b14-09e7-4b83-b442-98dbe009738d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:00.536410 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a020b14-09e7-4b83-b442-98dbe009738d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:00.536697 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a020b14-09e7-4b83-b442-98dbe009738d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:00.536976 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a020b14-09e7-4b83-b442-98dbe009738d None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:00.537697 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a020b14-09e7-4b83-b442-98dbe009738d None admin] RBAC: Adding query filter params (name=member) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:00.538003 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a020b14-09e7-4b83-b442-98dbe009738d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:00.539387 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a020b14-09e7-4b83-b442-98dbe009738d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:00.551118 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 206/419] 10.222.0.97 () {56 vars in 1101 bytes} [Tue Jul 26 14:30:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:00.556880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c84cd3-239b-4073-9e4a-c860cd76694d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:00.559681 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d70b2-f5fd-4418-8eb9-e2962163d4da None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:00.559681 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d70b2-f5fd-4418-8eb9-e2962163d4da None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:00.559681 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d70b2-f5fd-4418-8eb9-e2962163d4da None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:00.561108 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 207/420] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:00.566787 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91663a-c5ce-438a-b73a-3848291fd61d None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:00.567074 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91663a-c5ce-438a-b73a-3848291fd61d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:00.567351 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91663a-c5ce-438a-b73a-3848291fd61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:00.598506 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ad91663a-c5ce-438a-b73a-3848291fd61d None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:00.599882 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ad91663a-c5ce-438a-b73a-3848291fd61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:00.601488 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:00.708758 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 208/421] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 26 14:30:00.731067 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c84cd3-239b-4073-9e4a-c860cd76694d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:00.736775 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad91663a-c5ce-438a-b73a-3848291fd61d None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:00.746312 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:00.769915 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de2029-3a1c-4c5e-b571-3809b8a69cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:00.770549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de2029-3a1c-4c5e-b571-3809b8a69cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:00.770830 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de2029-3a1c-4c5e-b571-3809b8a69cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:00.771105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de2029-3a1c-4c5e-b571-3809b8a69cb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:00.887033 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de2029-3a1c-4c5e-b571-3809b8a69cb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:00.888513 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de2029-3a1c-4c5e-b571-3809b8a69cb0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:00.896630 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad91663a-c5ce-438a-b73a-3848291fd61d None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:00.947330 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:00.971169 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1eef13-6c17-4d8e-b978-5d023c8025e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:00.971863 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1eef13-6c17-4d8e-b978-5d023c8025e7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:00.972146 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1eef13-6c17-4d8e-b978-5d023c8025e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:00.972505 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1eef13-6c17-4d8e-b978-5d023c8025e7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:00.992671 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1eef13-6c17-4d8e-b978-5d023c8025e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:00.998153 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-5e1eef13-6c17-4d8e-b978-5d023c8025e7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:30:00.998153 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:00.998153 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:00.998153 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:00.998153 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:00.998153 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:00.998153 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:00.998153 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:00.998153 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:00.998153 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:00.998153 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:00.998153 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:00.998153 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:00.998153 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:00.998153 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:00.998153 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:00.999015 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:00.999015 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:00.999015 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:00.999015 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:00.999015 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:00.999015 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:00.999015 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:00.999015 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:00.999015 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:00.999015 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:00.999015 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:00.999015 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:00.999015 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:00.999015 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:00.999015 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:00.999015 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:00.999793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:00.999793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:00.999793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:00.999793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:00.999793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:00.999793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:00.999793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:00.999793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:00.999793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:00.999793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:00.999793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:30:00.999793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:01.007991 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 209/422] 10.222.0.97 () {56 vars in 1114 bytes} [Tue Jul 26 14:30:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:01.015433 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1eef13-6c17-4d8e-b978-5d023c8025e7 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:01.052592 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 214/423] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:30:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:30:01.060012 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de2029-3a1c-4c5e-b571-3809b8a69cb0 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:01.112081 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:01.144597 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1eef13-6c17-4d8e-b978-5d023c8025e7 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:01.180933 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c568ed6-3df2-4fa3-946d-203a5881b491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:01.181556 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c568ed6-3df2-4fa3-946d-203a5881b491 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:01.181838 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c568ed6-3df2-4fa3-946d-203a5881b491 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:01.182114 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c568ed6-3df2-4fa3-946d-203a5881b491 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:01.182861 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c568ed6-3df2-4fa3-946d-203a5881b491 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:01.183167 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c568ed6-3df2-4fa3-946d-203a5881b491 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:01.184602 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c568ed6-3df2-4fa3-946d-203a5881b491 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:01.193467 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 210/424] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:30:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:30:01.206841 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c568ed6-3df2-4fa3-946d-203a5881b491 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:01.234432 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de2029-3a1c-4c5e-b571-3809b8a69cb0 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:01.265071 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:01.290817 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3edd32-f06b-4926-924b-2bc74503c616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:01.293144 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3edd32-f06b-4926-924b-2bc74503c616 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:01.293670 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3edd32-f06b-4926-924b-2bc74503c616 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:01.294142 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3edd32-f06b-4926-924b-2bc74503c616 None admin] PATH_INFO: `/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:01.295604 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3edd32-f06b-4926-924b-2bc74503c616 None admin] RBAC: Authorizing `identity:get_project(project_id=076f7405c1874c1ea0d0f44c11ae4d38)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:01.304272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3edd32-f06b-4926-924b-2bc74503c616 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:01.306350 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 215/425] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:30:01 2022] GET /identity/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38 => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:30:01.312345 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71891dd4-7baf-4691-bdcf-40f056bf2394 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:01.312669 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71891dd4-7baf-4691-bdcf-40f056bf2394 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:01.313017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71891dd4-7baf-4691-bdcf-40f056bf2394 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:01.314978 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 216/426] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:01.315776 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71891dd4-7baf-4691-bdcf-40f056bf2394 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:01.340847 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c568ed6-3df2-4fa3-946d-203a5881b491 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:01.369703 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:01.378216 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c40521-906a-488b-b92b-dea63f54c212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:01.378925 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c40521-906a-488b-b92b-dea63f54c212 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:01.379216 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c40521-906a-488b-b92b-dea63f54c212 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:01.379526 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c40521-906a-488b-b92b-dea63f54c212 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:01.444164 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71891dd4-7baf-4691-bdcf-40f056bf2394 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:01.453625 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c40521-906a-488b-b92b-dea63f54c212 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:01.455218 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c40521-906a-488b-b92b-dea63f54c212 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:01.489206 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc946c4-fb46-4260-aeda-b485cc2a5491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:01.490125 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc946c4-fb46-4260-aeda-b485cc2a5491 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:01.490413 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc946c4-fb46-4260-aeda-b485cc2a5491 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:01.490743 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc946c4-fb46-4260-aeda-b485cc2a5491 None admin] PATH_INFO: `/v3/groups/5c4345316deb4e418266f43290528567` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:01.492209 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc946c4-fb46-4260-aeda-b485cc2a5491 None admin] RBAC: Authorizing `identity:get_group(group_id=5c4345316deb4e418266f43290528567)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:01.494615 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc946c4-fb46-4260-aeda-b485cc2a5491 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:01.496750 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 217/427] 10.222.0.97 () {56 vars in 1180 bytes} [Tue Jul 26 14:30:01 2022] GET /identity/v3/groups/5c4345316deb4e418266f43290528567 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:30:01.498500 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d9c7c-4d46-4287-a6a5-6731df71cd56 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:01.498785 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d9c7c-4d46-4287-a6a5-6731df71cd56 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:01.499064 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d9c7c-4d46-4287-a6a5-6731df71cd56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:01.524505 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-246d9c7c-4d46-4287-a6a5-6731df71cd56 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:01.525715 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-246d9c7c-4d46-4287-a6a5-6731df71cd56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:01.582489 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 211/428] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:30:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 14:30:01.583608 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c40521-906a-488b-b92b-dea63f54c212 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:01.602537 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:01.615494 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:01.617132 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 218/429] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:30:01.617989 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d9c7c-4d46-4287-a6a5-6731df71cd56 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:01.717258 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c40521-906a-488b-b92b-dea63f54c212 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:01.741433 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d9c7c-4d46-4287-a6a5-6731df71cd56 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:01.755351 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb797c74-635a-4af8-9dfb-cf453c0da5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:01.756027 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb797c74-635a-4af8-9dfb-cf453c0da5ea None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:01.756313 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb797c74-635a-4af8-9dfb-cf453c0da5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:01.756592 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb797c74-635a-4af8-9dfb-cf453c0da5ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:01.757427 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb797c74-635a-4af8-9dfb-cf453c0da5ea None admin] RBAC: Adding query filter params (group.id=5c4345316deb4e418266f43290528567, role.id=e033961a9363456a8510c34414dd839e, scope.project.id=076f7405c1874c1ea0d0f44c11ae4d38) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:01.757842 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb797c74-635a-4af8-9dfb-cf453c0da5ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:01.759605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb797c74-635a-4af8-9dfb-cf453c0da5ea None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:01.766236 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 212/430] 10.222.0.97 () {56 vars in 1376 bytes} [Tue Jul 26 14:30:01 2022] GET /identity/v3/role_assignments?group.id=5c4345316deb4e418266f43290528567&scope.project.id=076f7405c1874c1ea0d0f44c11ae4d38&role.id=e033961a9363456a8510c34414dd839e => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:01.767291 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb797c74-635a-4af8-9dfb-cf453c0da5ea None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:01.791153 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ff16c-1a19-44fe-b5b1-3094f569d1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:01.793140 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ff16c-1a19-44fe-b5b1-3094f569d1bf None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:01.794345 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ff16c-1a19-44fe-b5b1-3094f569d1bf None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:01.794640 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ff16c-1a19-44fe-b5b1-3094f569d1bf None admin] PATH_INFO: `/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:01.807939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21ff16c-1a19-44fe-b5b1-3094f569d1bf None admin] RBAC: Authorizing `identity:get_project(project_id=076f7405c1874c1ea0d0f44c11ae4d38)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:01.807939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21ff16c-1a19-44fe-b5b1-3094f569d1bf None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:01.808751 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 219/431] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:30:01 2022] GET /identity/v3/projects/076f7405c1874c1ea0d0f44c11ae4d38 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:01.816455 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ff16c-1a19-44fe-b5b1-3094f569d1bf None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:01.919176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:01.942093 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb797c74-635a-4af8-9dfb-cf453c0da5ea None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:01.962871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:02.005685 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee4a86e-167f-4159-91a1-4170b0647d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:02.006536 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4a86e-167f-4159-91a1-4170b0647d0f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:02.006823 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4a86e-167f-4159-91a1-4170b0647d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:02.007371 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4a86e-167f-4159-91a1-4170b0647d0f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:02.021891 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ff16c-1a19-44fe-b5b1-3094f569d1bf None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:02.023947 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee4a86e-167f-4159-91a1-4170b0647d0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:02.033596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-aee4a86e-167f-4159-91a1-4170b0647d0f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:30:02.033596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:02.033596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:02.033596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:02.033596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:02.033596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:02.033596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:02.033596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:02.033596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:02.033596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:02.033596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:02.033596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:02.033596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:02.033596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:02.033596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:02.033596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:02.034518 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:02.034518 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:02.034518 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:02.034518 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:02.034518 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:02.034518 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:02.034518 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:02.034518 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:02.034518 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:02.034518 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:02.034518 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:02.034518 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:02.034518 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:02.034518 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:02.034518 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:02.034518 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:02.035285 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:02.035285 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:02.035285 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:02.035285 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:02.035285 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:02.035285 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:02.035285 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:02.035285 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:02.035285 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:02.035285 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:02.035285 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:30:02.035285 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:02.035285 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 213/432] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:30:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:02.041338 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee4a86e-167f-4159-91a1-4170b0647d0f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:02.093938 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96795ce4-3733-4892-a589-b8e1878e6eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:02.094595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96795ce4-3733-4892-a589-b8e1878e6eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:02.094879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96795ce4-3733-4892-a589-b8e1878e6eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:02.095156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96795ce4-3733-4892-a589-b8e1878e6eb3 None admin] PATH_INFO: `/v3/groups/5c4345316deb4e418266f43290528567` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:02.096431 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96795ce4-3733-4892-a589-b8e1878e6eb3 None admin] RBAC: Authorizing `identity:get_group(group_id=5c4345316deb4e418266f43290528567)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:02.097829 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96795ce4-3733-4892-a589-b8e1878e6eb3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:02.099789 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 220/433] 10.222.0.97 () {56 vars in 1180 bytes} [Tue Jul 26 14:30:01 2022] GET /identity/v3/groups/5c4345316deb4e418266f43290528567 => generated 226 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:02.110354 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96795ce4-3733-4892-a589-b8e1878e6eb3 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:02.191140 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee4a86e-167f-4159-91a1-4170b0647d0f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:02.232946 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c117651-7a4a-4c30-81f5-09d89079e1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:02.234049 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c117651-7a4a-4c30-81f5-09d89079e1b5 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:02.234582 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c117651-7a4a-4c30-81f5-09d89079e1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:02.235089 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c117651-7a4a-4c30-81f5-09d89079e1b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:02.236736 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c117651-7a4a-4c30-81f5-09d89079e1b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:02.237311 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c117651-7a4a-4c30-81f5-09d89079e1b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:02.239257 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c117651-7a4a-4c30-81f5-09d89079e1b5 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:02.241973 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96795ce4-3733-4892-a589-b8e1878e6eb3 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:02.250073 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 214/434] 10.222.0.97 () {56 vars in 1101 bytes} [Tue Jul 26 14:30:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:30:02.258567 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c117651-7a4a-4c30-81f5-09d89079e1b5 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:02.282022 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:02.287382 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:02.296727 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b355adc-c215-45e6-b972-a3325e523a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:02.297684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b355adc-c215-45e6-b972-a3325e523a09 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:02.298622 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b355adc-c215-45e6-b972-a3325e523a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:02.301458 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b355adc-c215-45e6-b972-a3325e523a09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:02.301458 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b355adc-c215-45e6-b972-a3325e523a09 None admin] RBAC: Adding query filter params (group.id=5c4345316deb4e418266f43290528567, role.id=0602cf371b0f41b390e8a4499ae6bb3c, scope.project.id=076f7405c1874c1ea0d0f44c11ae4d38) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:02.301458 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b355adc-c215-45e6-b972-a3325e523a09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:02.302458 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b355adc-c215-45e6-b972-a3325e523a09 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:02.309882 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 221/435] 10.222.0.97 () {56 vars in 1376 bytes} [Tue Jul 26 14:30:02 2022] GET /identity/v3/role_assignments?group.id=5c4345316deb4e418266f43290528567&scope.project.id=076f7405c1874c1ea0d0f44c11ae4d38&role.id=0602cf371b0f41b390e8a4499ae6bb3c => generated 611 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:02.311221 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f14be6-4d6c-4a8e-8237-cde995d9daa6 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:02.311563 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f14be6-4d6c-4a8e-8237-cde995d9daa6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:02.311840 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f14be6-4d6c-4a8e-8237-cde995d9daa6 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:02.313258 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 222/436] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:02.319227 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c07cce-aa7d-4bf2-aeb3-249733d7cea7 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:02.319590 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c07cce-aa7d-4bf2-aeb3-249733d7cea7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:02.319749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c07cce-aa7d-4bf2-aeb3-249733d7cea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:02.338595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-02c07cce-aa7d-4bf2-aeb3-249733d7cea7 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:02.340352 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-02c07cce-aa7d-4bf2-aeb3-249733d7cea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:02.402292 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c117651-7a4a-4c30-81f5-09d89079e1b5 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:02.423029 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 223/437] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:30:02.443587 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18c3cd-268e-439c-8d70-7881a546e3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:02.444606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18c3cd-268e-439c-8d70-7881a546e3ea None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:02.445096 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18c3cd-268e-439c-8d70-7881a546e3ea None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:02.445348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c07cce-aa7d-4bf2-aeb3-249733d7cea7 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:02.446136 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18c3cd-268e-439c-8d70-7881a546e3ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:02.534796 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e18c3cd-268e-439c-8d70-7881a546e3ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:02.536914 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e18c3cd-268e-439c-8d70-7881a546e3ea None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:02.560382 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:02.585957 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c07cce-aa7d-4bf2-aeb3-249733d7cea7 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:02.623302 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:02.641953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd54116e-c895-49a6-b59a-d47e0916bb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:02.642909 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd54116e-c895-49a6-b59a-d47e0916bb54 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:02.643201 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd54116e-c895-49a6-b59a-d47e0916bb54 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:02.643610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd54116e-c895-49a6-b59a-d47e0916bb54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:02.657660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd54116e-c895-49a6-b59a-d47e0916bb54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:02.663065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-bd54116e-c895-49a6-b59a-d47e0916bb54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:30:02.663065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:02.663065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:02.663065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:02.663065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:02.663065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:02.663065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:02.663065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:02.663065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:02.663065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:02.663065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:02.663065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:02.663065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:02.663065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:02.663065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:02.663065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:02.663944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:02.663944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:02.663944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:02.663944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:02.663944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:02.663944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:02.663944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:02.663944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:02.663944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:02.663944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:02.663944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:02.663944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:02.663944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:02.663944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:02.663944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:02.663944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:02.664711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:02.664711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:02.664711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:02.664711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:02.664711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:02.664711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:02.664711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:02.664711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:02.664711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:02.664711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:02.664711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:30:02.664711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:02.664711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 224/438] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:30:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:30:02.670467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd54116e-c895-49a6-b59a-d47e0916bb54 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:02.687040 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 215/439] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:30:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:30:02.693430 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18c3cd-268e-439c-8d70-7881a546e3ea None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:02.801980 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd54116e-c895-49a6-b59a-d47e0916bb54 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:02.823079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18c3cd-268e-439c-8d70-7881a546e3ea None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:02.843342 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:02.849519 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817fbc9-646f-4ce6-b4f0-e15343e48f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:02.850408 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817fbc9-646f-4ce6-b4f0-e15343e48f5d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:02.850696 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817fbc9-646f-4ce6-b4f0-e15343e48f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:02.850975 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817fbc9-646f-4ce6-b4f0-e15343e48f5d None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:02.851763 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817fbc9-646f-4ce6-b4f0-e15343e48f5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:02.852072 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817fbc9-646f-4ce6-b4f0-e15343e48f5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:02.853645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817fbc9-646f-4ce6-b4f0-e15343e48f5d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:02.864875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 225/440] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:30:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:02.869543 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad25f28b-6f0f-434d-ae67-6490d248363e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:02.869543 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25f28b-6f0f-434d-ae67-6490d248363e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:02.869543 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25f28b-6f0f-434d-ae67-6490d248363e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:02.870683 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25f28b-6f0f-434d-ae67-6490d248363e None admin] PATH_INFO: `/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:02.871531 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817fbc9-646f-4ce6-b4f0-e15343e48f5d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:02.871912 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad25f28b-6f0f-434d-ae67-6490d248363e None admin] RBAC: Authorizing `identity:get_project(project_id=5dbf1af61d63414cbf4c33bdc412a9d6)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:02.873363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad25f28b-6f0f-434d-ae67-6490d248363e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:02.879749 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 216/441] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:30:02 2022] GET /identity/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:30:02.881074 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad25f28b-6f0f-434d-ae67-6490d248363e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:02.929490 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:02.986463 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817fbc9-646f-4ce6-b4f0-e15343e48f5d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:02.991976 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad25f28b-6f0f-434d-ae67-6490d248363e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:03.023080 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c331620-656c-400d-b579-27b1f61aba19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:03.023734 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c331620-656c-400d-b579-27b1f61aba19 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:03.024021 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c331620-656c-400d-b579-27b1f61aba19 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:03.024300 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c331620-656c-400d-b579-27b1f61aba19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:03.025872 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced198c-bbf2-425b-9442-45e77c2f08d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:03.027125 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced198c-bbf2-425b-9442-45e77c2f08d2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:03.027424 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced198c-bbf2-425b-9442-45e77c2f08d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:03.027732 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced198c-bbf2-425b-9442-45e77c2f08d2 None admin] PATH_INFO: `/v3/groups/5c4345316deb4e418266f43290528567` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:03.028894 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced198c-bbf2-425b-9442-45e77c2f08d2 None admin] RBAC: Authorizing `identity:get_group(group_id=5c4345316deb4e418266f43290528567)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:03.030315 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced198c-bbf2-425b-9442-45e77c2f08d2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:03.031970 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 217/442] 10.222.0.97 () {56 vars in 1180 bytes} [Tue Jul 26 14:30:02 2022] GET /identity/v3/groups/5c4345316deb4e418266f43290528567 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:30:03.033070 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96d5a9-05eb-4a83-952d-9deeb64a21af None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:03.033363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96d5a9-05eb-4a83-952d-9deeb64a21af None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:03.033636 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96d5a9-05eb-4a83-952d-9deeb64a21af None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:03.034852 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 218/443] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:03.038831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96d5a9-05eb-4a83-952d-9deeb64a21af None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:03.090675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:03.096157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c331620-656c-400d-b579-27b1f61aba19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:03.097621 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c331620-656c-400d-b579-27b1f61aba19 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:03.141058 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96d5a9-05eb-4a83-952d-9deeb64a21af None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:03.142559 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:03.175253 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63205dca-8173-4ac3-8342-584933a20d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:03.175891 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63205dca-8173-4ac3-8342-584933a20d2f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:03.176177 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63205dca-8173-4ac3-8342-584933a20d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:03.176487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63205dca-8173-4ac3-8342-584933a20d2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:03.177268 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63205dca-8173-4ac3-8342-584933a20d2f None admin] RBAC: Adding query filter params (group.id=5c4345316deb4e418266f43290528567, role.id=e033961a9363456a8510c34414dd839e, scope.project.id=5dbf1af61d63414cbf4c33bdc412a9d6) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:03.177578 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63205dca-8173-4ac3-8342-584933a20d2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:03.178991 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63205dca-8173-4ac3-8342-584933a20d2f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:03.185235 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 219/444] 10.222.0.97 () {56 vars in 1376 bytes} [Tue Jul 26 14:30:03 2022] GET /identity/v3/role_assignments?group.id=5c4345316deb4e418266f43290528567&scope.project.id=5dbf1af61d63414cbf4c33bdc412a9d6&role.id=e033961a9363456a8510c34414dd839e => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:03.186436 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448061c9-225d-4847-8260-9b83d9ddd5c2 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:03.186729 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448061c9-225d-4847-8260-9b83d9ddd5c2 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:03.187011 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448061c9-225d-4847-8260-9b83d9ddd5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:03.203764 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 226/445] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:30:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:30:03.204631 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c331620-656c-400d-b579-27b1f61aba19 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:03.210259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-448061c9-225d-4847-8260-9b83d9ddd5c2 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:03.211633 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-448061c9-225d-4847-8260-9b83d9ddd5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:03.294259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 220/446] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:30:03.314508 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c331620-656c-400d-b579-27b1f61aba19 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:03.317999 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448061c9-225d-4847-8260-9b83d9ddd5c2 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:03.356997 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d502a7-67d2-4c94-b17f-d02be44adfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:03.357654 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d502a7-67d2-4c94-b17f-d02be44adfe7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:03.357944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d502a7-67d2-4c94-b17f-d02be44adfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:03.358221 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d502a7-67d2-4c94-b17f-d02be44adfe7 None admin] PATH_INFO: `/v3/projects/011581630bbe4c60a042395d12032595` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:03.359619 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d502a7-67d2-4c94-b17f-d02be44adfe7 None admin] RBAC: Authorizing `identity:get_project(project_id=011581630bbe4c60a042395d12032595)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:03.361054 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d502a7-67d2-4c94-b17f-d02be44adfe7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:03.362878 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 227/447] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:30:03 2022] GET /identity/v3/projects/011581630bbe4c60a042395d12032595 => generated 344 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:03.368764 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d502a7-67d2-4c94-b17f-d02be44adfe7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:03.375269 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:03.387763 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:03.440579 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448061c9-225d-4847-8260-9b83d9ddd5c2 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:03.481483 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e13bd-10c4-40a6-bbea-a27b13c7b576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:03.482537 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e13bd-10c4-40a6-bbea-a27b13c7b576 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:03.482821 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e13bd-10c4-40a6-bbea-a27b13c7b576 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:03.483104 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e13bd-10c4-40a6-bbea-a27b13c7b576 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:03.490467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d502a7-67d2-4c94-b17f-d02be44adfe7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:03.498656 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e13bd-10c4-40a6-bbea-a27b13c7b576 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:03.503172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-639e13bd-10c4-40a6-bbea-a27b13c7b576 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:30:03.503172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:03.503172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:03.503172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:03.503172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:03.503172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:03.503172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:03.503172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:03.503172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:03.503172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:03.503172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:03.503172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:03.503172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:03.503172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:03.503172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:03.503172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:03.504147 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:03.504147 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:03.504147 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:03.504147 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:03.504147 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:03.504147 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:03.504147 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:03.504147 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:03.504147 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:03.504147 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:03.504147 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:03.504147 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:03.504147 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:03.504147 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:03.504147 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:03.504147 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:03.504930 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:03.504930 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:03.504930 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:03.504930 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:03.504930 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:03.504930 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:03.504930 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:03.504930 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:03.504930 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:03.504930 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:03.504930 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:30:03.504930 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:03.504930 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 221/448] 10.222.0.97 () {56 vars in 1114 bytes} [Tue Jul 26 14:30:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:30:03.510901 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e13bd-10c4-40a6-bbea-a27b13c7b576 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:03.535345 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa9abc-70e6-46aa-b786-7791446e39d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:03.536081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa9abc-70e6-46aa-b786-7791446e39d7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:03.536363 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa9abc-70e6-46aa-b786-7791446e39d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:03.536671 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa9abc-70e6-46aa-b786-7791446e39d7 None admin] PATH_INFO: `/v3/groups/71acea0c5f2f489e957b341cd3328328` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:03.538932 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa9abc-70e6-46aa-b786-7791446e39d7 None admin] RBAC: Authorizing `identity:get_group(group_id=71acea0c5f2f489e957b341cd3328328)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:03.539312 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa9abc-70e6-46aa-b786-7791446e39d7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:03.541120 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 228/449] 10.222.0.97 () {56 vars in 1180 bytes} [Tue Jul 26 14:30:03 2022] GET /identity/v3/groups/71acea0c5f2f489e957b341cd3328328 => generated 229 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:30:03.549408 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa9abc-70e6-46aa-b786-7791446e39d7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:03.618243 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e13bd-10c4-40a6-bbea-a27b13c7b576 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:03.620545 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:03.628995 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:03.647735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa9abc-70e6-46aa-b786-7791446e39d7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:03.651526 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed0101-a1fd-4876-9f9e-bffd180de81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:03.652145 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed0101-a1fd-4876-9f9e-bffd180de81c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:03.652431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed0101-a1fd-4876-9f9e-bffd180de81c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:03.652714 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed0101-a1fd-4876-9f9e-bffd180de81c None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:03.653675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ed0101-a1fd-4876-9f9e-bffd180de81c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:03.654131 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ed0101-a1fd-4876-9f9e-bffd180de81c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:03.655826 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ed0101-a1fd-4876-9f9e-bffd180de81c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:03.663693 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 222/450] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:30:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:30:03.669687 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed0101-a1fd-4876-9f9e-bffd180de81c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:03.681637 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aefb294-3d81-408b-800a-e3ac26c7fcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:03.682279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefb294-3d81-408b-800a-e3ac26c7fcff None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:03.682598 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefb294-3d81-408b-800a-e3ac26c7fcff None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:03.682870 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefb294-3d81-408b-800a-e3ac26c7fcff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:03.683681 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aefb294-3d81-408b-800a-e3ac26c7fcff None admin] RBAC: Adding query filter params (group.id=71acea0c5f2f489e957b341cd3328328, role.id=51b2d10dfffb4b75b186f3d8b15e2a78, scope.project.id=011581630bbe4c60a042395d12032595) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:03.683995 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aefb294-3d81-408b-800a-e3ac26c7fcff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:03.685424 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aefb294-3d81-408b-800a-e3ac26c7fcff None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:03.691060 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 229/451] 10.222.0.97 () {56 vars in 1376 bytes} [Tue Jul 26 14:30:03 2022] GET /identity/v3/role_assignments?group.id=71acea0c5f2f489e957b341cd3328328&scope.project.id=011581630bbe4c60a042395d12032595&role.id=51b2d10dfffb4b75b186f3d8b15e2a78 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:03.774574 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed0101-a1fd-4876-9f9e-bffd180de81c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:03.809869 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a690a-059a-4627-a81a-7aaf3dce8ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:03.810498 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a690a-059a-4627-a81a-7aaf3dce8ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:03.810785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a690a-059a-4627-a81a-7aaf3dce8ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:03.811063 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a690a-059a-4627-a81a-7aaf3dce8ee3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:03.847438 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:03.882229 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a690a-059a-4627-a81a-7aaf3dce8ee3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:03.883634 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a690a-059a-4627-a81a-7aaf3dce8ee3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:03.978648 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 223/452] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:30:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 14:30:03.984296 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aefb294-3d81-408b-800a-e3ac26c7fcff None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:04.081121 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aefb294-3d81-408b-800a-e3ac26c7fcff None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:04.114144 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e9c22-5e55-409d-ac9c-d015791a8752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:04.114777 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e9c22-5e55-409d-ac9c-d015791a8752 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:04.115063 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e9c22-5e55-409d-ac9c-d015791a8752 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:04.115337 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e9c22-5e55-409d-ac9c-d015791a8752 None admin] PATH_INFO: `/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:04.116513 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e9c22-5e55-409d-ac9c-d015791a8752 None admin] RBAC: Authorizing `identity:get_project(project_id=5dbf1af61d63414cbf4c33bdc412a9d6)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:04.118006 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e9c22-5e55-409d-ac9c-d015791a8752 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:04.119683 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 230/453] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:30:03 2022] GET /identity/v3/projects/5dbf1af61d63414cbf4c33bdc412a9d6 => generated 302 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:04.125626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a690a-059a-4627-a81a-7aaf3dce8ee3 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:04.189160 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:04.221667 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a690a-059a-4627-a81a-7aaf3dce8ee3 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:04.254311 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b0f425-eb40-4087-ab02-8f39ec3aeeaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:04.254941 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0f425-eb40-4087-ab02-8f39ec3aeeaf None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:04.255227 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0f425-eb40-4087-ab02-8f39ec3aeeaf None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:04.255534 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0f425-eb40-4087-ab02-8f39ec3aeeaf None admin] PATH_INFO: `/v3/groups/5c4345316deb4e418266f43290528567` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:04.256666 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b0f425-eb40-4087-ab02-8f39ec3aeeaf None admin] RBAC: Authorizing `identity:get_group(group_id=5c4345316deb4e418266f43290528567)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:04.258095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b0f425-eb40-4087-ab02-8f39ec3aeeaf None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:04.259691 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 224/454] 10.222.0.97 () {56 vars in 1180 bytes} [Tue Jul 26 14:30:04 2022] GET /identity/v3/groups/5c4345316deb4e418266f43290528567 => generated 226 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:04.265928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e9c22-5e55-409d-ac9c-d015791a8752 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:04.280642 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:04.362550 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e9c22-5e55-409d-ac9c-d015791a8752 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:04.395192 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6a645-e25b-4884-85ec-d8918c5b637e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:04.395825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6a645-e25b-4884-85ec-d8918c5b637e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:04.396112 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6a645-e25b-4884-85ec-d8918c5b637e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:04.396390 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6a645-e25b-4884-85ec-d8918c5b637e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:04.397452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a6a645-e25b-4884-85ec-d8918c5b637e None admin] RBAC: Adding query filter params (group.id=5c4345316deb4e418266f43290528567, role.id=0602cf371b0f41b390e8a4499ae6bb3c, scope.project.id=5dbf1af61d63414cbf4c33bdc412a9d6) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:04.397765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a6a645-e25b-4884-85ec-d8918c5b637e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:04.399180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a6a645-e25b-4884-85ec-d8918c5b637e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:04.405321 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 231/455] 10.222.0.97 () {56 vars in 1376 bytes} [Tue Jul 26 14:30:04 2022] GET /identity/v3/role_assignments?group.id=5c4345316deb4e418266f43290528567&scope.project.id=5dbf1af61d63414cbf4c33bdc412a9d6&role.id=0602cf371b0f41b390e8a4499ae6bb3c => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:07.062246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3235e9-9bc9-41e1-88b8-9154d993fc1f None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:07.062685 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3235e9-9bc9-41e1-88b8-9154d993fc1f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:07.062805 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3235e9-9bc9-41e1-88b8-9154d993fc1f None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:07.064693 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 225/456] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:07.077428 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a6c5d-e5b6-4af6-b1bd-0003b1c1b8b0 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:07.078417 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a6c5d-e5b6-4af6-b1bd-0003b1c1b8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:07.078893 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a6c5d-e5b6-4af6-b1bd-0003b1c1b8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:07.108382 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-4c0a6c5d-e5b6-4af6-b1bd-0003b1c1b8b0 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:07.109937 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4c0a6c5d-e5b6-4af6-b1bd-0003b1c1b8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:07.188125 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2437487-65d8-4df3-99a2-173ac311071d None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:07.188773 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2437487-65d8-4df3-99a2-173ac311071d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:07.189263 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2437487-65d8-4df3-99a2-173ac311071d None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:07.190793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 226/457] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:30:07.196937 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af0bf6-9d39-4af6-8d73-084553e4a72a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:07.196937 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af0bf6-9d39-4af6-8d73-084553e4a72a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:07.196937 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af0bf6-9d39-4af6-8d73-084553e4a72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:07.215767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 232/458] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:30:07.222632 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-39af0bf6-9d39-4af6-8d73-084553e4a72a None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:07.223873 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-39af0bf6-9d39-4af6-8d73-084553e4a72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:07.245122 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a6c5d-e5b6-4af6-b1bd-0003b1c1b8b0 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:07.254753 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:07.368170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 227/459] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:30:07.380506 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a6c5d-e5b6-4af6-b1bd-0003b1c1b8b0 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:07.424991 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af0bf6-9d39-4af6-8d73-084553e4a72a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:07.436851 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61cd03-4c83-4da5-b572-08cd9ab8ecad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:07.437515 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61cd03-4c83-4da5-b572-08cd9ab8ecad None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:07.437797 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61cd03-4c83-4da5-b572-08cd9ab8ecad None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:07.438252 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61cd03-4c83-4da5-b572-08cd9ab8ecad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:07.451370 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61cd03-4c83-4da5-b572-08cd9ab8ecad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:07.452882 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61cd03-4c83-4da5-b572-08cd9ab8ecad None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:07.471156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-8e61cd03-4c83-4da5-b572-08cd9ab8ecad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:07.471156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:07.471156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:07.471156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:07.471156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:07.471156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:07.471156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:07.471156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:07.471156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:07.471156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:07.471156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:07.471156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:07.471156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:07.471156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:07.471156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:07.472242 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:07.472242 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:07.472242 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:07.472242 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:07.472242 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:07.472242 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:07.472242 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:07.472242 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:07.472242 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:07.472242 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:07.472242 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:07.472242 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:07.472242 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:07.472242 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:07.472242 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:07.472242 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:07.472242 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:07.473279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:07.473279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:07.473279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:07.473279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:07.475320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 233/460] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:30:07.476789 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d58d926-7460-49c9-8362-3721ee9628c0 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:07.477217 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d58d926-7460-49c9-8362-3721ee9628c0 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:07.477634 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d58d926-7460-49c9-8362-3721ee9628c0 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:07.479094 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 234/461] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:07.484826 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ec958-7402-44f1-8d10-6be8eee4d817 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:07.485296 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ec958-7402-44f1-8d10-6be8eee4d817 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:07.485810 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ec958-7402-44f1-8d10-6be8eee4d817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:07.496693 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:07.506498 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-7e7ec958-7402-44f1-8d10-6be8eee4d817 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:07.507998 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7e7ec958-7402-44f1-8d10-6be8eee4d817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:07.535868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af0bf6-9d39-4af6-8d73-084553e4a72a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:07.571217 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c497b5c-9b07-49ab-b598-4ac628c78610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:07.571725 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:07.572514 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c497b5c-9b07-49ab-b598-4ac628c78610 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:07.573129 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c497b5c-9b07-49ab-b598-4ac628c78610 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:07.573418 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c497b5c-9b07-49ab-b598-4ac628c78610 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:07.584969 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c497b5c-9b07-49ab-b598-4ac628c78610 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:07.586423 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c497b5c-9b07-49ab-b598-4ac628c78610 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:07.586941 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 235/462] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:30:07.588062 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ec958-7402-44f1-8d10-6be8eee4d817 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:07.604657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-3c497b5c-9b07-49ab-b598-4ac628c78610 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:07.604657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:07.604657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:07.604657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:07.604657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:07.604657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:07.604657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:07.604657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:07.604657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:07.604657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:07.604657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:07.604657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:07.604657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:07.604657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:07.604657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:07.605510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:07.605510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:07.605510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:07.605510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:07.605510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:07.605510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:07.605510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:07.605510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:07.605510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:07.605510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:07.605510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:07.605510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:07.605510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:07.605510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:07.605510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:07.605510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:07.605510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:07.606347 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:07.606347 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:07.606347 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:07.606347 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:07.606548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 228/463] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:30:07.613021 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c497b5c-9b07-49ab-b598-4ac628c78610 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:07.717734 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ec958-7402-44f1-8d10-6be8eee4d817 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:07.747812 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c497b5c-9b07-49ab-b598-4ac628c78610 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:07.753391 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29720436-6bb6-4c02-9aa4-f7f406254775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:07.754136 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29720436-6bb6-4c02-9aa4-f7f406254775 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:07.754420 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29720436-6bb6-4c02-9aa4-f7f406254775 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:07.754697 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29720436-6bb6-4c02-9aa4-f7f406254775 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:07.755383 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29720436-6bb6-4c02-9aa4-f7f406254775 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:07.755715 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29720436-6bb6-4c02-9aa4-f7f406254775 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:07.757312 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29720436-6bb6-4c02-9aa4-f7f406254775 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:07.764959 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:07.769474 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 236/464] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:07.769973 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29720436-6bb6-4c02-9aa4-f7f406254775 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:07.789475 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b837967-7d2b-4c3c-a4f4-e5ef398c36b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:07.790421 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b837967-7d2b-4c3c-a4f4-e5ef398c36b4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:07.790928 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b837967-7d2b-4c3c-a4f4-e5ef398c36b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:07.791967 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b837967-7d2b-4c3c-a4f4-e5ef398c36b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:07.793396 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b837967-7d2b-4c3c-a4f4-e5ef398c36b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:07.794248 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b837967-7d2b-4c3c-a4f4-e5ef398c36b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:07.796560 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b837967-7d2b-4c3c-a4f4-e5ef398c36b4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:07.813431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 229/465] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:07.815686 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefefa7-6248-4220-9251-fd1b8398fe64 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:07.816185 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefefa7-6248-4220-9251-fd1b8398fe64 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:07.816921 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefefa7-6248-4220-9251-fd1b8398fe64 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:07.819390 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 230/466] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:07.820608 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:07.821127 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefefa7-6248-4220-9251-fd1b8398fe64 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:07.874357 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29720436-6bb6-4c02-9aa4-f7f406254775 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:07.912027 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416035e6-7826-46d3-92ad-d48069497fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:07.912672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416035e6-7826-46d3-92ad-d48069497fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:07.912961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416035e6-7826-46d3-92ad-d48069497fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:07.913241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416035e6-7826-46d3-92ad-d48069497fd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:07.924795 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416035e6-7826-46d3-92ad-d48069497fd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:07.926297 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416035e6-7826-46d3-92ad-d48069497fd2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:07.941867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-416035e6-7826-46d3-92ad-d48069497fd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:07.941867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:07.941867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:07.941867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:07.941867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:07.941867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:07.941867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:07.941867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:07.941867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:07.941867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:07.941867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:07.941867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:07.941867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:07.941867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:07.941867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:07.942764 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:07.942764 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:07.942764 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:07.942764 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:07.942764 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:07.942764 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:07.942764 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:07.942764 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:07.942764 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:07.942764 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:07.942764 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:07.942764 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:07.942764 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:07.942764 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:07.942764 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:07.942764 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:07.942764 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:07.943612 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:07.943612 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:07.943612 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:07.943612 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:07.943612 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 237/467] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:07.944417 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416035e6-7826-46d3-92ad-d48069497fd2 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:07.951306 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefefa7-6248-4220-9251-fd1b8398fe64 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:07.990026 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282bc006-6439-4088-9270-df33b428de4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:07.990894 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282bc006-6439-4088-9270-df33b428de4c None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:07.991185 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282bc006-6439-4088-9270-df33b428de4c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:07.991503 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282bc006-6439-4088-9270-df33b428de4c None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:07.992186 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282bc006-6439-4088-9270-df33b428de4c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:07.993641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282bc006-6439-4088-9270-df33b428de4c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:08.047131 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:08.050464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416035e6-7826-46d3-92ad-d48069497fd2 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:08.083355 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81302fa8-037b-49e9-8c7c-5a3a954b8957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:08.084014 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81302fa8-037b-49e9-8c7c-5a3a954b8957 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:08.084298 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81302fa8-037b-49e9-8c7c-5a3a954b8957 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:08.084576 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81302fa8-037b-49e9-8c7c-5a3a954b8957 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:08.085248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81302fa8-037b-49e9-8c7c-5a3a954b8957 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:08.086678 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81302fa8-037b-49e9-8c7c-5a3a954b8957 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:08.145080 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 231/468] 10.222.0.97 () {60 vars in 1132 bytes} [Tue Jul 26 14:30:07 2022] POST /identity/v3/users => generated 246 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:30:08.146309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d449e-97e0-4ece-ae5f-6f6cccb8e125 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:08.146596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d449e-97e0-4ece-ae5f-6f6cccb8e125 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:08.146873 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d449e-97e0-4ece-ae5f-6f6cccb8e125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:08.168587 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-047d449e-97e0-4ece-ae5f-6f6cccb8e125 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:08.169886 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-047d449e-97e0-4ece-ae5f-6f6cccb8e125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:08.199045 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:08.247383 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 238/469] 10.222.0.97 () {60 vars in 1132 bytes} [Tue Jul 26 14:30:07 2022] POST /identity/v3/users => generated 248 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:30:08.247383 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81302fa8-037b-49e9-8c7c-5a3a954b8957 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:08.247676 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 232/470] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:30:08.247676 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0467397-d2ab-4ee7-8f01-fce535d89422 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:08.247676 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0467397-d2ab-4ee7-8f01-fce535d89422 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:08.247676 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0467397-d2ab-4ee7-8f01-fce535d89422 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:08.247909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 233/471] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:08.258033 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6e42d-54a0-40a6-a4ba-0921daa5acf8 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:08.258371 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6e42d-54a0-40a6-a4ba-0921daa5acf8 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:08.258606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6e42d-54a0-40a6-a4ba-0921daa5acf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:08.289597 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-46d6e42d-54a0-40a6-a4ba-0921daa5acf8 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:08.290852 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-46d6e42d-54a0-40a6-a4ba-0921daa5acf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:08.387618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 234/472] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:30:08.388852 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d6e42d-54a0-40a6-a4ba-0921daa5acf8 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:08.413058 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81302fa8-037b-49e9-8c7c-5a3a954b8957 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:08.461031 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41a749-3b42-4a05-a9ac-8c2e987b372e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:08.461721 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41a749-3b42-4a05-a9ac-8c2e987b372e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:08.462007 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41a749-3b42-4a05-a9ac-8c2e987b372e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:08.462392 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41a749-3b42-4a05-a9ac-8c2e987b372e None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:08.463103 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41a749-3b42-4a05-a9ac-8c2e987b372e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:08.463580 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41a749-3b42-4a05-a9ac-8c2e987b372e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:08.465047 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41a749-3b42-4a05-a9ac-8c2e987b372e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:08.484052 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 239/473] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:08.484645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41a749-3b42-4a05-a9ac-8c2e987b372e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:08.508604 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:08.519544 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:08.530612 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d6e42d-54a0-40a6-a4ba-0921daa5acf8 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:08.571342 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c5066-052a-4e02-807c-f0f0f868e444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:08.572088 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c5066-052a-4e02-807c-f0f0f868e444 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:08.572377 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c5066-052a-4e02-807c-f0f0f868e444 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:08.573195 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c5066-052a-4e02-807c-f0f0f868e444 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:08.584583 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927c5066-052a-4e02-807c-f0f0f868e444 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:08.586070 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927c5066-052a-4e02-807c-f0f0f868e444 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:08.601491 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41a749-3b42-4a05-a9ac-8c2e987b372e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:08.602768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-927c5066-052a-4e02-807c-f0f0f868e444 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:08.602768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:08.602768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:08.602768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:08.602768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:08.602768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:08.602768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:08.602768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:08.602768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:08.602768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:08.602768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:08.602768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:08.602768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:08.602768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:08.602768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:08.603605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:08.603605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:08.603605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:08.603605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:08.603605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:08.603605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:08.603605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:08.603605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:08.603605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:08.603605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:08.603605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:08.603605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:08.603605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:08.603605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:08.603605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:08.603605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:08.603605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:08.604961 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:08.604961 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:08.604961 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:08.604961 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:08.609705 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 235/474] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:08.610626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c5066-052a-4e02-807c-f0f0f868e444 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:08.645171 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841b94b-b156-4767-a9e5-b2fce60dba5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:08.645799 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841b94b-b156-4767-a9e5-b2fce60dba5c None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:08.646086 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841b94b-b156-4767-a9e5-b2fce60dba5c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:08.646358 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841b94b-b156-4767-a9e5-b2fce60dba5c None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:08.647255 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841b94b-b156-4767-a9e5-b2fce60dba5c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:08.649519 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841b94b-b156-4767-a9e5-b2fce60dba5c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:08.693598 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.sql.core [None req-9841b94b-b156-4767-a9e5-b2fce60dba5c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 26 14:30:08.693598 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 26 14:30:08.693598 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [parameters: {'id': '029360d9190747cfbc1851ce0b2b4f0d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 26 14:30:08.693598 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80523) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 26 14:30:08.721048 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-9841b94b-b156-4767-a9e5-b2fce60dba5c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 26 14:30:08.721048 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:08.721048 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 14:30:08.721048 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 14:30:08.721048 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 14:30:08.721048 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 14:30:08.721048 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 14:30:08.721048 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 14:30:08.721048 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 14:30:08.721048 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application conn.query(q) Jul 26 14:30:08.721048 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 14:30:08.721048 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 14:30:08.721048 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 14:30:08.721048 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application result.read() Jul 26 14:30:08.721048 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 14:30:08.721983 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 14:30:08.721983 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 14:30:08.721983 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 14:30:08.721983 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 14:30:08.721983 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 14:30:08.721983 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 14:30:08.721983 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 14:30:08.721983 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 26 14:30:08.721983 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:08.721983 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 26 14:30:08.721983 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:08.721983 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:08.721983 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 26 14:30:08.721983 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 26 14:30:08.721983 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 26 14:30:08.721983 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return ref.to_dict() Jul 26 14:30:08.721983 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 14:30:08.721983 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application next(self.gen) Jul 26 14:30:08.722871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 26 14:30:08.722871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application yield resource Jul 26 14:30:08.722871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 14:30:08.722871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application next(self.gen) Jul 26 14:30:08.722871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 26 14:30:08.722871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self.session.rollback() Jul 26 14:30:08.722871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 26 14:30:08.722871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self.force_reraise() Jul 26 14:30:08.722871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 26 14:30:08.722871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise self.value Jul 26 14:30:08.722871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 26 14:30:08.722871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 26 14:30:08.722871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 26 14:30:08.722871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application session.commit() Jul 26 14:30:08.722871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 26 14:30:08.722871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 26 14:30:08.722871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 26 14:30:08.723825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._prepare_impl() Jul 26 14:30:08.723825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 26 14:30:08.723825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self.session.flush() Jul 26 14:30:08.723825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 26 14:30:08.723825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._flush(objects) Jul 26 14:30:08.723825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 26 14:30:08.723825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 26 14:30:08.723825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 26 14:30:08.723825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application compat.raise_( Jul 26 14:30:08.723825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 14:30:08.723825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception Jul 26 14:30:08.723825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 26 14:30:08.723825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application flush_context.execute() Jul 26 14:30:08.723825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 26 14:30:08.723825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rec.execute(self) Jul 26 14:30:08.723825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 26 14:30:08.723825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 26 14:30:08.724684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 26 14:30:08.724684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 26 14:30:08.724684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 26 14:30:08.724684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 26 14:30:08.724684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 26 14:30:08.724684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 26 14:30:08.724684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 26 14:30:08.724684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 26 14:30:08.724684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 26 14:30:08.724684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 26 14:30:08.724684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 26 14:30:08.724684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 26 14:30:08.724684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 26 14:30:08.724684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 26 14:30:08.724684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 14:30:08.724684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception Jul 26 14:30:08.725478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 14:30:08.725478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 14:30:08.725478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 14:30:08.725478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 14:30:08.725478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 14:30:08.725478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 14:30:08.725478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 14:30:08.725478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application conn.query(q) Jul 26 14:30:08.725478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 14:30:08.725478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 14:30:08.725478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 14:30:08.725478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application result.read() Jul 26 14:30:08.725478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 14:30:08.725478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 14:30:08.725478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 14:30:08.725478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 14:30:08.726256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 14:30:08.726256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 14:30:08.726256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 14:30:08.726256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 14:30:08.726256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 26 14:30:08.726256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 26 14:30:08.726256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [parameters: {'id': '029360d9190747cfbc1851ce0b2b4f0d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 26 14:30:08.726256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 26 14:30:08.726256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:08.726256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:30:08.726256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:08.726256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:08.726256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:08.726256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:08.726256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:08.726256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:08.727028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:08.727028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:08.727028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:08.727028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:08.727028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:08.727028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:08.727028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 26 14:30:08.727028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 26 14:30:08.727028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:08.727028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:08.727028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 26 14:30:08.727028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 26 14:30:08.727028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 26 14:30:08.727028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 26 14:30:08.727028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 26 14:30:08.727028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:08.732319 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 240/475] 10.222.0.97 () {60 vars in 1131 bytes} [Tue Jul 26 14:30:08 2022] POST /identity/v3/roles => generated 144 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:08.732404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c5066-052a-4e02-807c-f0f0f868e444 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:08.732488 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841b94b-b156-4767-a9e5-b2fce60dba5c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:08.763752 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d2b4d-813b-4ce9-b3cd-35eb9e67c293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:08.763752 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d2b4d-813b-4ce9-b3cd-35eb9e67c293 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:08.763752 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d2b4d-813b-4ce9-b3cd-35eb9e67c293 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:08.763752 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d2b4d-813b-4ce9-b3cd-35eb9e67c293 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:08.764428 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d2b4d-813b-4ce9-b3cd-35eb9e67c293 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:08.766024 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d2b4d-813b-4ce9-b3cd-35eb9e67c293 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:08.820446 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:08.842435 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841b94b-b156-4767-a9e5-b2fce60dba5c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:08.863478 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:08.869721 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 236/476] 10.222.0.97 () {60 vars in 1132 bytes} [Tue Jul 26 14:30:08 2022] POST /identity/v3/users => generated 248 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:30:08.871687 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d2b4d-813b-4ce9-b3cd-35eb9e67c293 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:08.884228 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e4e05-cf9c-43be-b7d3-6c329c64338d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:08.885178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e4e05-cf9c-43be-b7d3-6c329c64338d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:08.885550 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e4e05-cf9c-43be-b7d3-6c329c64338d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:08.885876 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e4e05-cf9c-43be-b7d3-6c329c64338d None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:08.886655 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e4e05-cf9c-43be-b7d3-6c329c64338d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:08.886966 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e4e05-cf9c-43be-b7d3-6c329c64338d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:08.889293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e4e05-cf9c-43be-b7d3-6c329c64338d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:08.913074 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 241/477] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:08.915102 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e4e05-cf9c-43be-b7d3-6c329c64338d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:09.032654 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d2b4d-813b-4ce9-b3cd-35eb9e67c293 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:09.093108 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6455e10-83df-4f24-abf8-dde774c842fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:09.094163 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6455e10-83df-4f24-abf8-dde774c842fa None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:09.094656 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6455e10-83df-4f24-abf8-dde774c842fa None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:09.095139 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6455e10-83df-4f24-abf8-dde774c842fa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:09.110004 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6455e10-83df-4f24-abf8-dde774c842fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:09.119179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-a6455e10-83df-4f24-abf8-dde774c842fa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:30:09.119179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:09.119179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:09.119179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:09.119179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:09.119179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:09.119179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:09.119179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:09.119179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:09.119179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:09.119179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:09.119179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:09.119179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:09.119179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:09.119179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:09.119179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:09.120161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:09.120161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:09.120161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:09.120161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:09.120161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:09.120161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:09.120161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:09.120161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:09.120161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:09.120161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:09.120161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:09.120161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:09.120161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:09.120161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:09.120161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:09.120161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:09.120962 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:09.120962 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:09.120962 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:09.120962 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:09.120962 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:09.120962 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:09.120962 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:09.120962 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:09.120962 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:09.120962 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:09.120962 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:30:09.120962 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:09.123288 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 237/478] 10.222.0.97 () {56 vars in 1114 bytes} [Tue Jul 26 14:30:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:09.130110 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6455e10-83df-4f24-abf8-dde774c842fa None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:09.145208 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e4e05-cf9c-43be-b7d3-6c329c64338d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:09.197116 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37819a-f13b-4525-9486-517f65ba42fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:09.198400 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37819a-f13b-4525-9486-517f65ba42fd None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:09.198748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37819a-f13b-4525-9486-517f65ba42fd None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:09.199029 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37819a-f13b-4525-9486-517f65ba42fd None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:09.199808 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a37819a-f13b-4525-9486-517f65ba42fd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:09.202427 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a37819a-f13b-4525-9486-517f65ba42fd None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:09.219946 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:09.266124 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6455e10-83df-4f24-abf8-dde774c842fa None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:09.301188 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707937ef-e389-4a0f-8dc2-c95ef64b885b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:09.301763 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707937ef-e389-4a0f-8dc2-c95ef64b885b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:09.302051 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707937ef-e389-4a0f-8dc2-c95ef64b885b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:09.302333 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707937ef-e389-4a0f-8dc2-c95ef64b885b None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:09.303052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707937ef-e389-4a0f-8dc2-c95ef64b885b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:09.303372 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707937ef-e389-4a0f-8dc2-c95ef64b885b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:09.304820 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707937ef-e389-4a0f-8dc2-c95ef64b885b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:09.314465 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 238/479] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:30:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:09.459791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:09.547227 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 242/480] 10.222.0.97 () {60 vars in 1132 bytes} [Tue Jul 26 14:30:08 2022] POST /identity/v3/users => generated 249 bytes in 632 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:30:10.488449 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e19a70-3f59-4bd8-8912-612c6bec9dd2 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:10.488914 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e19a70-3f59-4bd8-8912-612c6bec9dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:10.488914 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e19a70-3f59-4bd8-8912-612c6bec9dd2 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:10.490574 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 239/481] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:10.495996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631d8f8-4552-4f95-9e1b-69176c2546b5 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:10.496406 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631d8f8-4552-4f95-9e1b-69176c2546b5 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:10.496578 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631d8f8-4552-4f95-9e1b-69176c2546b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:10.524494 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6631d8f8-4552-4f95-9e1b-69176c2546b5 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:10.526234 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6631d8f8-4552-4f95-9e1b-69176c2546b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:10.533273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49e9a9-3063-4fc4-82da-c3920d6027ee None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:10.533576 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49e9a9-3063-4fc4-82da-c3920d6027ee None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:10.533832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49e9a9-3063-4fc4-82da-c3920d6027ee None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:10.535535 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 240/482] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:10.543235 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f7ce3-7791-4764-98a8-445e6504e7b3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:10.543589 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f7ce3-7791-4764-98a8-445e6504e7b3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:10.543835 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f7ce3-7791-4764-98a8-445e6504e7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:10.578365 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-fc6f7ce3-7791-4764-98a8-445e6504e7b3 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:10.579614 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fc6f7ce3-7791-4764-98a8-445e6504e7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:10.665999 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 243/483] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:30:10.694091 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 241/484] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:30:10.700789 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631d8f8-4552-4f95-9e1b-69176c2546b5 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:10.752390 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f7ce3-7791-4764-98a8-445e6504e7b3 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:10.845807 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631d8f8-4552-4f95-9e1b-69176c2546b5 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:10.851751 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:10.876830 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:10.893326 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d81e2-80bf-4d2d-8304-fa6c08c4a016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:10.894273 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d81e2-80bf-4d2d-8304-fa6c08c4a016 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:10.894783 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d81e2-80bf-4d2d-8304-fa6c08c4a016 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:10.895262 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d81e2-80bf-4d2d-8304-fa6c08c4a016 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:10.914893 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f7ce3-7791-4764-98a8-445e6504e7b3 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:10.917638 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16d81e2-80bf-4d2d-8304-fa6c08c4a016 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:10.931423 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-b16d81e2-80bf-4d2d-8304-fa6c08c4a016 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:10.931423 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:10.931423 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:10.931423 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:10.931423 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:10.931423 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:10.931423 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:10.931423 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:10.931423 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:10.931423 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:10.931423 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:10.931423 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:10.931423 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:10.931423 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:10.931423 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:10.931423 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:10.932347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:10.932347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:10.932347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:10.932347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:10.932347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:10.932347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:10.932347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:10.932347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:10.932347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:10.932347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:10.932347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:10.932347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:10.932347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:10.932347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:10.932347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:10.932347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:10.933116 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:10.933116 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:10.933116 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:10.933116 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:10.933116 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:10.933116 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:10.933116 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:10.933116 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:10.933116 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:10.933116 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:10.933116 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:10.933116 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:10.935736 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 244/485] 10.222.0.97 () {56 vars in 1102 bytes} [Tue Jul 26 14:30:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:10.943201 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d81e2-80bf-4d2d-8304-fa6c08c4a016 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:10.950303 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b7f67-f064-46ea-b87e-08f11227768b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:10.950959 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b7f67-f064-46ea-b87e-08f11227768b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:10.951250 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b7f67-f064-46ea-b87e-08f11227768b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:10.951605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b7f67-f064-46ea-b87e-08f11227768b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:10.962155 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30b7f67-f064-46ea-b87e-08f11227768b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:10.967861 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-f30b7f67-f064-46ea-b87e-08f11227768b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:10.967861 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:10.967861 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:10.967861 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:10.967861 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:10.967861 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:10.967861 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:10.967861 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:10.967861 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:10.967861 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:10.967861 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:10.967861 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:10.967861 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:10.967861 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:10.967861 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:10.967861 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:10.968730 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:10.968730 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:10.968730 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:10.968730 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:10.968730 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:10.968730 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:10.968730 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:10.968730 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:10.968730 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:10.968730 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:10.968730 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:10.968730 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:10.968730 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:10.968730 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:10.968730 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:10.968730 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:10.969516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:10.969516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:10.969516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:10.969516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:10.969516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:10.969516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:10.969516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:10.969516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:10.969516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:10.969516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:10.969516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:10.969516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:10.969516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 242/486] 10.222.0.97 () {56 vars in 1102 bytes} [Tue Jul 26 14:30:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:30:10.984455 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b7f67-f064-46ea-b87e-08f11227768b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:11.109386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d81e2-80bf-4d2d-8304-fa6c08c4a016 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:11.181731 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b7f67-f064-46ea-b87e-08f11227768b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:11.183972 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae61d1af-8a27-44c0-9729-3f1a681f0b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:11.184607 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61d1af-8a27-44c0-9729-3f1a681f0b0d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:11.184886 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61d1af-8a27-44c0-9729-3f1a681f0b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:11.185156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61d1af-8a27-44c0-9729-3f1a681f0b0d None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:11.185883 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae61d1af-8a27-44c0-9729-3f1a681f0b0d None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:11.186188 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae61d1af-8a27-44c0-9729-3f1a681f0b0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:11.187627 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae61d1af-8a27-44c0-9729-3f1a681f0b0d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:11.198881 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 245/487] 10.222.0.97 () {56 vars in 1103 bytes} [Tue Jul 26 14:30:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:30:11.204612 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae61d1af-8a27-44c0-9729-3f1a681f0b0d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:11.236128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:11.275555 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f2ab8-c62f-4c35-b2a1-ee0f63140855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:11.276182 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f2ab8-c62f-4c35-b2a1-ee0f63140855 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:11.276467 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f2ab8-c62f-4c35-b2a1-ee0f63140855 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:11.276745 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f2ab8-c62f-4c35-b2a1-ee0f63140855 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:11.277472 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f2ab8-c62f-4c35-b2a1-ee0f63140855 None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:11.277777 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f2ab8-c62f-4c35-b2a1-ee0f63140855 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:11.279150 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f2ab8-c62f-4c35-b2a1-ee0f63140855 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:11.296271 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 243/488] 10.222.0.97 () {56 vars in 1103 bytes} [Tue Jul 26 14:30:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:11.302120 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f2ab8-c62f-4c35-b2a1-ee0f63140855 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:11.303763 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:11.398499 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae61d1af-8a27-44c0-9729-3f1a681f0b0d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:11.460210 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b255ebe-9e6c-439f-a66b-5479550664b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:11.460867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b255ebe-9e6c-439f-a66b-5479550664b4 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:11.461150 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b255ebe-9e6c-439f-a66b-5479550664b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:11.461428 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b255ebe-9e6c-439f-a66b-5479550664b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:11.475691 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b255ebe-9e6c-439f-a66b-5479550664b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:11.477095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f2ab8-c62f-4c35-b2a1-ee0f63140855 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:11.477207 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b255ebe-9e6c-439f-a66b-5479550664b4 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:11.508434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-7b255ebe-9e6c-439f-a66b-5479550664b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:11.508434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:11.508434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:11.508434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:11.508434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:11.508434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:11.508434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:11.508434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:11.508434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:11.508434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:11.508434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:11.508434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:11.508434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:11.508434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:11.508434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:11.509782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:11.509782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:11.509782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:11.509782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:11.509782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:11.509782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:11.509782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:11.509782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:11.509782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:11.509782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:11.509782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:11.509782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:11.509782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:11.509782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:11.509782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:11.509782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:11.509782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:11.510625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:11.510625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:11.510625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:11.510625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:11.510625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 246/489] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:11.530248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b255ebe-9e6c-439f-a66b-5479550664b4 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:11.550381 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b036c814-4a46-4006-a140-a320ee27b35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:11.551029 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036c814-4a46-4006-a140-a320ee27b35c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:11.551314 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036c814-4a46-4006-a140-a320ee27b35c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:11.551618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036c814-4a46-4006-a140-a320ee27b35c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:11.564013 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b036c814-4a46-4006-a140-a320ee27b35c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:11.565470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b036c814-4a46-4006-a140-a320ee27b35c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:11.581012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-b036c814-4a46-4006-a140-a320ee27b35c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:11.581012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:11.581012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:11.581012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:11.581012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:11.581012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:11.581012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:11.581012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:11.581012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:11.581012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:11.581012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:11.581012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:11.581012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:11.581012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:11.581012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:11.582764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:11.582764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:11.582764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:11.582764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:11.582764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:11.582764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:11.582764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:11.582764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:11.582764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:11.582764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:11.582764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:11.582764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:11.582764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:11.582764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:11.582764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:11.582764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:11.582764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:11.584178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:11.584178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:11.584178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:11.584178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:11.584178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 244/490] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:30:11.590500 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b036c814-4a46-4006-a140-a320ee27b35c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:11.628769 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:11.673818 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:11.676943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b255ebe-9e6c-439f-a66b-5479550664b4 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:11.712823 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b036c814-4a46-4006-a140-a320ee27b35c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:11.714388 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ed064-51c5-4a0e-aa79-26c5be128775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:11.715018 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ed064-51c5-4a0e-aa79-26c5be128775 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:11.715305 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ed064-51c5-4a0e-aa79-26c5be128775 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:11.715627 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ed064-51c5-4a0e-aa79-26c5be128775 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:11.716333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ed064-51c5-4a0e-aa79-26c5be128775 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:11.716638 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ed064-51c5-4a0e-aa79-26c5be128775 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:11.718036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ed064-51c5-4a0e-aa79-26c5be128775 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:11.734981 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 247/491] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:11.734981 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311fef6-7b59-41c9-8d4f-0ab1903d1952 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:11.734981 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311fef6-7b59-41c9-8d4f-0ab1903d1952 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:11.734981 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311fef6-7b59-41c9-8d4f-0ab1903d1952 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:11.734981 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 248/492] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:11.734981 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e5868-926c-4af8-bea7-732cd852c85b None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:11.735543 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e5868-926c-4af8-bea7-732cd852c85b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:11.736607 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e5868-926c-4af8-bea7-732cd852c85b None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:11.736915 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 249/493] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:11.737605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e5868-926c-4af8-bea7-732cd852c85b None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:11.754592 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843dc423-4736-4536-afdc-6e444df15921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:11.755213 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843dc423-4736-4536-afdc-6e444df15921 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:11.755549 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843dc423-4736-4536-afdc-6e444df15921 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:11.755830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843dc423-4736-4536-afdc-6e444df15921 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:11.756511 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843dc423-4736-4536-afdc-6e444df15921 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:11.756821 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843dc423-4736-4536-afdc-6e444df15921 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:11.758188 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843dc423-4736-4536-afdc-6e444df15921 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:11.772842 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 245/494] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:11.774993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8800d4-2c4c-4c3e-9974-6640787068f8 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:11.776105 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8800d4-2c4c-4c3e-9974-6640787068f8 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:11.776714 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8800d4-2c4c-4c3e-9974-6640787068f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:11.798565 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-4a8800d4-2c4c-4c3e-9974-6640787068f8 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:11.800091 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4a8800d4-2c4c-4c3e-9974-6640787068f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:11.852804 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e5868-926c-4af8-bea7-732cd852c85b None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:11.870974 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 246/495] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:30:11.872398 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80815285-b971-494f-9c24-e277b33930bc None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:11.872687 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80815285-b971-494f-9c24-e277b33930bc None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:11.872974 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80815285-b971-494f-9c24-e277b33930bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:11.892950 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-80815285-b971-494f-9c24-e277b33930bc None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:11.894126 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-80815285-b971-494f-9c24-e277b33930bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:11.901934 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354f56b-1ccf-4ee1-aca4-ae90bbbaac13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:11.903002 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354f56b-1ccf-4ee1-aca4-ae90bbbaac13 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:11.903288 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354f56b-1ccf-4ee1-aca4-ae90bbbaac13 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:11.903609 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354f56b-1ccf-4ee1-aca4-ae90bbbaac13 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:11.906831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:11.912387 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354f56b-1ccf-4ee1-aca4-ae90bbbaac13 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:11.913839 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354f56b-1ccf-4ee1-aca4-ae90bbbaac13 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:11.927845 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-e354f56b-1ccf-4ee1-aca4-ae90bbbaac13 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:30:11.927845 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:11.927845 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:11.927845 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:11.927845 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:11.927845 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:11.927845 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:11.927845 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:11.927845 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:11.927845 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:11.927845 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:11.927845 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:11.927845 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:30:11.927845 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:30:11.927845 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:30:11.927845 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:30:11.928890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:11.928890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:11.928890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:30:11.928890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:11.928890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:30:11.928890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:11.928890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:11.928890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:11.928890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:11.928890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:11.928890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:11.928890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:11.928890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:11.928890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:11.928890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:11.928890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:11.928890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:11.928890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:11.929776 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:11.929776 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:11.929776 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:30:11.929776 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:30:11.929776 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:30:11.929776 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:30:11.929776 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:30:11.929776 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:30:11.929776 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:30:11.929776 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:11.929776 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 250/496] 10.222.0.97 () {56 vars in 1093 bytes} [Tue Jul 26 14:30:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:11.930358 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354f56b-1ccf-4ee1-aca4-ae90bbbaac13 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:11.957828 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:11.973634 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 247/497] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:30:11.974379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80815285-b971-494f-9c24-e277b33930bc None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:12.060971 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354f56b-1ccf-4ee1-aca4-ae90bbbaac13 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:12.087599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80815285-b971-494f-9c24-e277b33930bc None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:12.096069 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23619e10-eafd-40c6-862c-3076104bebfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:12.096809 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23619e10-eafd-40c6-862c-3076104bebfe None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:12.097087 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23619e10-eafd-40c6-862c-3076104bebfe None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:12.097357 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23619e10-eafd-40c6-862c-3076104bebfe None admin] PATH_INFO: `/v3/users/glance` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:12.105203 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23619e10-eafd-40c6-862c-3076104bebfe None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:12.106659 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23619e10-eafd-40c6-862c-3076104bebfe None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:12.118034 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-23619e10-eafd-40c6-862c-3076104bebfe None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 26 14:30:12.118034 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:12.118034 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:12.118034 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:12.118034 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:12.118034 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:12.118034 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:12.118034 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:12.118034 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:12.118034 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:12.118034 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:12.118034 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:12.118034 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:30:12.118034 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:30:12.118034 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:30:12.118034 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:30:12.118923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:12.118923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:12.118923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:30:12.118923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:12.118923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:30:12.118923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:12.118923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:12.118923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:12.118923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:12.118923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:12.118923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:12.118923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:12.118923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:12.118923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:12.118923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:12.118923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:12.118923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:12.118923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:12.120385 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:12.120385 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:12.120385 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:30:12.120385 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:30:12.120385 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:30:12.120385 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:30:12.120385 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:30:12.120385 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:30:12.120385 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 26 14:30:12.120385 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:12.120385 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 251/498] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:30:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:12.120951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23619e10-eafd-40c6-862c-3076104bebfe None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:12.129482 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727103f-1eb5-4f5e-bd35-abd72bae4490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:12.130490 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727103f-1eb5-4f5e-bd35-abd72bae4490 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:12.130490 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727103f-1eb5-4f5e-bd35-abd72bae4490 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:12.130679 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727103f-1eb5-4f5e-bd35-abd72bae4490 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:12.142883 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1727103f-1eb5-4f5e-bd35-abd72bae4490 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:12.148377 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-1727103f-1eb5-4f5e-bd35-abd72bae4490 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:12.148377 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:12.148377 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:12.148377 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:12.148377 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:12.148377 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:12.148377 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:12.148377 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:12.148377 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:12.148377 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:12.148377 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:12.148377 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:12.148377 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:12.148377 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:12.148377 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:12.148377 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:12.149249 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:12.149249 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:12.149249 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:12.149249 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:12.149249 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:12.149249 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:12.149249 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:12.149249 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:12.149249 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:12.149249 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:12.149249 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:12.149249 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:12.149249 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:12.149249 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:12.149249 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:12.149249 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:12.150009 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:12.150009 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:12.150009 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:12.150009 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:12.150009 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:12.150009 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:12.150009 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:12.150009 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:12.150009 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:12.150009 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:12.150009 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:12.150009 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:12.150009 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 248/499] 10.222.0.97 () {56 vars in 1102 bytes} [Tue Jul 26 14:30:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:12.150684 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727103f-1eb5-4f5e-bd35-abd72bae4490 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:12.159797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:12.209626 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:12.228080 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23619e10-eafd-40c6-862c-3076104bebfe None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:12.250458 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727103f-1eb5-4f5e-bd35-abd72bae4490 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:12.262033 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88540940-f312-42a6-adfd-25ede251ec4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:12.262654 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88540940-f312-42a6-adfd-25ede251ec4c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:12.262933 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88540940-f312-42a6-adfd-25ede251ec4c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:12.263206 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88540940-f312-42a6-adfd-25ede251ec4c None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:12.264054 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88540940-f312-42a6-adfd-25ede251ec4c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:12.264363 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88540940-f312-42a6-adfd-25ede251ec4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:12.265786 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88540940-f312-42a6-adfd-25ede251ec4c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:12.283949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d296ca-a1d9-47e6-a48d-d1358f515a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:12.284578 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d296ca-a1d9-47e6-a48d-d1358f515a55 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:12.284865 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d296ca-a1d9-47e6-a48d-d1358f515a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:12.285144 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d296ca-a1d9-47e6-a48d-d1358f515a55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:12.295015 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d296ca-a1d9-47e6-a48d-d1358f515a55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:12.296476 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d296ca-a1d9-47e6-a48d-d1358f515a55 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:12.301215 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 252/500] 10.222.0.97 () {56 vars in 1133 bytes} [Tue Jul 26 14:30:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:12.302020 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88540940-f312-42a6-adfd-25ede251ec4c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:12.314531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-43d296ca-a1d9-47e6-a48d-d1358f515a55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:12.314531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:12.314531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:12.314531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:12.314531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:12.314531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:12.314531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:12.314531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:12.314531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:12.314531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:12.314531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:12.314531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:12.314531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:12.314531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:12.314531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:12.315590 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:12.315590 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:12.315590 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:12.315590 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:12.315590 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:12.315590 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:12.315590 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:12.315590 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:12.315590 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:12.315590 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:12.315590 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:12.315590 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:12.315590 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:12.315590 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:12.315590 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:12.315590 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:12.315590 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:12.316806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:12.316806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:12.316806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:12.316806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:12.316806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 249/501] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:12.317095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff0d58-7aac-4f21-a8a9-9facbb6bbe2e None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:12.317095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff0d58-7aac-4f21-a8a9-9facbb6bbe2e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:12.317386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff0d58-7aac-4f21-a8a9-9facbb6bbe2e None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:12.318635 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 250/502] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:12.319325 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff0d58-7aac-4f21-a8a9-9facbb6bbe2e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:12.380281 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:12.409182 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88540940-f312-42a6-adfd-25ede251ec4c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:12.423645 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff0d58-7aac-4f21-a8a9-9facbb6bbe2e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:12.442970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc76178-13ce-41da-af00-b4a79bfa0628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:12.443623 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc76178-13ce-41da-af00-b4a79bfa0628 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:12.443907 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc76178-13ce-41da-af00-b4a79bfa0628 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:12.444184 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc76178-13ce-41da-af00-b4a79bfa0628 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:12.444977 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc76178-13ce-41da-af00-b4a79bfa0628 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:12.445286 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc76178-13ce-41da-af00-b4a79bfa0628 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:12.446723 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc76178-13ce-41da-af00-b4a79bfa0628 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:12.455116 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:12.457209 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49a6af-9faf-4f9d-a309-c6cd87e5b52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:12.457819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49a6af-9faf-4f9d-a309-c6cd87e5b52a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:12.458106 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49a6af-9faf-4f9d-a309-c6cd87e5b52a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:12.458382 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49a6af-9faf-4f9d-a309-c6cd87e5b52a None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:12.459127 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49a6af-9faf-4f9d-a309-c6cd87e5b52a None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:12.459463 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49a6af-9faf-4f9d-a309-c6cd87e5b52a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:12.460864 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49a6af-9faf-4f9d-a309-c6cd87e5b52a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:12.464885 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 253/503] 10.222.0.97 () {56 vars in 1137 bytes} [Tue Jul 26 14:30:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:12.465752 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc76178-13ce-41da-af00-b4a79bfa0628 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:12.473495 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 251/504] 10.222.0.97 () {56 vars in 1103 bytes} [Tue Jul 26 14:30:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:12.474318 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49a6af-9faf-4f9d-a309-c6cd87e5b52a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:12.570887 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc76178-13ce-41da-af00-b4a79bfa0628 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:12.579044 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49a6af-9faf-4f9d-a309-c6cd87e5b52a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:12.598614 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:12.608670 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21605a-b405-4534-938b-d5ffd8a3f874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:12.609322 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21605a-b405-4534-938b-d5ffd8a3f874 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:12.612259 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21605a-b405-4534-938b-d5ffd8a3f874 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:12.612259 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21605a-b405-4534-938b-d5ffd8a3f874 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:12.612860 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec807921-b0bf-4ecb-b103-933cccb750f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:12.613478 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec807921-b0bf-4ecb-b103-933cccb750f0 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:12.613767 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec807921-b0bf-4ecb-b103-933cccb750f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:12.614046 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec807921-b0bf-4ecb-b103-933cccb750f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:12.614741 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec807921-b0bf-4ecb-b103-933cccb750f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:12.615052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec807921-b0bf-4ecb-b103-933cccb750f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:12.616510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec807921-b0bf-4ecb-b103-933cccb750f0 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:12.628681 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 252/505] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:12.629855 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2101f-3757-4255-ac7e-f69e001dc523 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:12.630143 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2101f-3757-4255-ac7e-f69e001dc523 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:12.630419 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2101f-3757-4255-ac7e-f69e001dc523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:12.648397 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-1bb2101f-3757-4255-ac7e-f69e001dc523 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:12.649602 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1bb2101f-3757-4255-ac7e-f69e001dc523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:12.678507 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:12.685445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa21605a-b405-4534-938b-d5ffd8a3f874 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:12.686970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa21605a-b405-4534-938b-d5ffd8a3f874 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:12.717014 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 253/506] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:30:12.717784 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb2101f-3757-4255-ac7e-f69e001dc523 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:12.786623 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 254/507] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:30:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:30:12.787496 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21605a-b405-4534-938b-d5ffd8a3f874 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:12.828192 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb2101f-3757-4255-ac7e-f69e001dc523 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:12.839531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:12.861954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230b0f7-cbe6-4a62-9765-9f7bed8634ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:12.862569 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230b0f7-cbe6-4a62-9765-9f7bed8634ed None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:12.862853 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230b0f7-cbe6-4a62-9765-9f7bed8634ed None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:12.863135 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230b0f7-cbe6-4a62-9765-9f7bed8634ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:12.880457 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:12.886702 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21605a-b405-4534-938b-d5ffd8a3f874 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:12.920623 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e8fab3-75de-4a9f-8157-2967bc3dc3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:12.921264 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8fab3-75de-4a9f-8157-2967bc3dc3f7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:12.921550 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8fab3-75de-4a9f-8157-2967bc3dc3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:12.921827 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8fab3-75de-4a9f-8157-2967bc3dc3f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:12.932331 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e8fab3-75de-4a9f-8157-2967bc3dc3f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:12.933347 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230b0f7-cbe6-4a62-9765-9f7bed8634ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:12.933803 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e8fab3-75de-4a9f-8157-2967bc3dc3f7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:12.934782 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230b0f7-cbe6-4a62-9765-9f7bed8634ed None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:12.949035 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-b5e8fab3-75de-4a9f-8157-2967bc3dc3f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:12.949035 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:12.949035 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:12.949035 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:12.949035 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:12.949035 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:12.949035 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:12.949035 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:12.949035 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:12.949035 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:12.949035 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:12.949035 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:12.949035 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:12.949035 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:12.949035 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:12.949875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:12.949875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:12.949875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:12.949875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:12.949875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:12.949875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:12.949875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:12.949875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:12.949875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:12.949875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:12.949875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:12.949875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:12.949875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:12.949875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:12.949875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:12.949875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:12.949875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:12.950699 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:12.950699 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:12.950699 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:12.950699 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:12.950699 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 255/508] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:12.950958 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e8fab3-75de-4a9f-8157-2967bc3dc3f7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:13.040533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 254/509] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:30:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:30:13.041274 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230b0f7-cbe6-4a62-9765-9f7bed8634ed None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:13.053406 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:13.061178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e8fab3-75de-4a9f-8157-2967bc3dc3f7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:13.098288 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72bbf2c-2a3b-4356-9fc0-ff73e801a2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:13.098950 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72bbf2c-2a3b-4356-9fc0-ff73e801a2fc None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:13.099240 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72bbf2c-2a3b-4356-9fc0-ff73e801a2fc None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:13.099550 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72bbf2c-2a3b-4356-9fc0-ff73e801a2fc None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:13.100222 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72bbf2c-2a3b-4356-9fc0-ff73e801a2fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:13.101666 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72bbf2c-2a3b-4356-9fc0-ff73e801a2fc None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:13.141139 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230b0f7-cbe6-4a62-9765-9f7bed8634ed None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:13.173905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7dc256-785a-4480-8470-7d6e2d0bc255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:13.174542 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7dc256-785a-4480-8470-7d6e2d0bc255 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:13.174827 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7dc256-785a-4480-8470-7d6e2d0bc255 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:13.175103 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7dc256-785a-4480-8470-7d6e2d0bc255 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:13.185482 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7dc256-785a-4480-8470-7d6e2d0bc255 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:13.190456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-cd7dc256-785a-4480-8470-7d6e2d0bc255 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:13.190456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:13.190456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:13.190456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:13.190456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:13.190456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:13.190456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:13.190456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:13.190456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:13.190456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:13.190456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:13.190456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:13.190456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:13.190456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:13.190456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:13.190456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:13.191533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:13.191533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:13.191533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:13.191533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:13.191533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:13.191533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:13.191533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:13.191533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:13.191533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:13.191533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:13.191533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:13.191533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:13.191533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:13.191533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:13.191533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:13.191533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:13.192349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:13.192349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:13.192349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:13.192349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:13.192349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:13.192349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:13.192349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:13.192349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:13.192349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:13.192349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:13.192349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:13.192349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:13.192349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 255/510] 10.222.0.97 () {56 vars in 1102 bytes} [Tue Jul 26 14:30:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:13.192986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7dc256-785a-4480-8470-7d6e2d0bc255 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:13.207360 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 256/511] 10.222.0.97 () {60 vars in 1132 bytes} [Tue Jul 26 14:30:12 2022] POST /identity/v3/users => generated 247 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:30:13.224433 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72bbf2c-2a3b-4356-9fc0-ff73e801a2fc None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:13.231683 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:13.277031 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:13.315462 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7dc256-785a-4480-8470-7d6e2d0bc255 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:13.341028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72bbf2c-2a3b-4356-9fc0-ff73e801a2fc None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:13.356467 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09771457-39d5-4494-ad28-cf6e69a10159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:13.357298 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09771457-39d5-4494-ad28-cf6e69a10159 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:13.357594 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09771457-39d5-4494-ad28-cf6e69a10159 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:13.357864 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09771457-39d5-4494-ad28-cf6e69a10159 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:13.372332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09771457-39d5-4494-ad28-cf6e69a10159 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:13.374038 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09771457-39d5-4494-ad28-cf6e69a10159 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:13.384549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eccea12-895b-4ecf-b5fc-96cef23d4408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:13.386155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccea12-895b-4ecf-b5fc-96cef23d4408 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:13.386488 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccea12-895b-4ecf-b5fc-96cef23d4408 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:13.386765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccea12-895b-4ecf-b5fc-96cef23d4408 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:13.387505 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eccea12-895b-4ecf-b5fc-96cef23d4408 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:13.387817 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eccea12-895b-4ecf-b5fc-96cef23d4408 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:13.389203 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eccea12-895b-4ecf-b5fc-96cef23d4408 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:13.395030 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-09771457-39d5-4494-ad28-cf6e69a10159 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:13.395030 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:13.395030 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:13.395030 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:13.395030 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:13.395030 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:13.395030 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:13.395030 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:13.395030 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:13.395030 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:13.395030 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:13.395030 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:13.395030 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:13.395030 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:13.395030 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:13.395910 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:13.395910 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:13.395910 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:13.395910 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:13.395910 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:13.395910 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:13.395910 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:13.395910 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:13.395910 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:13.395910 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:13.395910 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:13.395910 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:13.395910 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:13.395910 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:13.395910 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:13.395910 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:13.395910 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:13.396748 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:13.396748 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:13.396748 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:13.396748 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:13.401039 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 256/512] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:13.402053 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09771457-39d5-4494-ad28-cf6e69a10159 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:13.402168 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 257/513] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:13.402754 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eccea12-895b-4ecf-b5fc-96cef23d4408 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:13.473357 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:13.520889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09771457-39d5-4494-ad28-cf6e69a10159 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:13.528838 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eccea12-895b-4ecf-b5fc-96cef23d4408 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:13.556577 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:13.572593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8db06f-d715-40a3-b930-6321add7826e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:13.573175 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8db06f-d715-40a3-b930-6321add7826e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:13.573456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8db06f-d715-40a3-b930-6321add7826e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:13.573734 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8db06f-d715-40a3-b930-6321add7826e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:13.584251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8db06f-d715-40a3-b930-6321add7826e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:13.585699 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8db06f-d715-40a3-b930-6321add7826e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:13.586524 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06831414-568d-47c2-a1ef-f5cb894cece5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:13.587175 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06831414-568d-47c2-a1ef-f5cb894cece5 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:13.587488 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06831414-568d-47c2-a1ef-f5cb894cece5 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:13.587756 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06831414-568d-47c2-a1ef-f5cb894cece5 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:13.589404 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06831414-568d-47c2-a1ef-f5cb894cece5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:13.589779 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06831414-568d-47c2-a1ef-f5cb894cece5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:13.591062 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06831414-568d-47c2-a1ef-f5cb894cece5 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:13.604030 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 258/514] 10.222.0.97 () {56 vars in 1103 bytes} [Tue Jul 26 14:30:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:13.604030 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06831414-568d-47c2-a1ef-f5cb894cece5 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:13.604251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-4a8db06f-d715-40a3-b930-6321add7826e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:13.604251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:13.604251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:13.604251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:13.604251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:13.604251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:13.604251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:13.604251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:13.604251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:13.604251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:13.604251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:13.604251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:13.604251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:13.604251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:13.604251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:13.605010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:13.605010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:13.605010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:13.605010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:13.605010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:13.605010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:13.605010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:13.605010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:13.605010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:13.605010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:13.605010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:13.605010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:13.605010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:13.605010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:13.605010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:13.605010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:13.605010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:13.605823 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:13.605823 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:13.605823 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:13.605823 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:13.605823 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 257/515] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:13.605823 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8db06f-d715-40a3-b930-6321add7826e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:13.712015 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8db06f-d715-40a3-b930-6321add7826e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:13.717515 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06831414-568d-47c2-a1ef-f5cb894cece5 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:13.743515 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:13.749950 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5b73c-d9b1-4504-892a-a1f900209493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:13.750631 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5b73c-d9b1-4504-892a-a1f900209493 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:13.750923 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5b73c-d9b1-4504-892a-a1f900209493 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:13.754321 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5b73c-d9b1-4504-892a-a1f900209493 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:13.754424 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48482287-c96c-420a-827c-cea4896666cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:13.754424 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48482287-c96c-420a-827c-cea4896666cc None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:13.754424 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48482287-c96c-420a-827c-cea4896666cc None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:13.754424 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48482287-c96c-420a-827c-cea4896666cc None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:13.754825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48482287-c96c-420a-827c-cea4896666cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:13.755131 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48482287-c96c-420a-827c-cea4896666cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:13.756553 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48482287-c96c-420a-827c-cea4896666cc None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:13.760965 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5b73c-d9b1-4504-892a-a1f900209493 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:13.762408 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5b73c-d9b1-4504-892a-a1f900209493 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:13.774245 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 259/516] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:13.774985 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48482287-c96c-420a-827c-cea4896666cc None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:13.776479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-52d5b73c-d9b1-4504-892a-a1f900209493 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 14:30:13.776479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:13.776479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:13.776479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:13.776479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:13.776479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:13.776479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:13.776479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:13.776479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:13.776479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:13.776479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:13.776479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:13.776479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:30:13.776479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:30:13.776479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:30:13.776479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:30:13.777604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:13.777604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:13.777604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:30:13.777604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:13.777604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:30:13.777604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:13.777604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:13.777604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:13.777604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:13.777604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:13.777604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:13.777604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:13.777604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:13.777604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:13.777604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:13.777604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:13.777604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:13.777604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:13.778472 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:13.778472 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:13.778472 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:30:13.778472 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:30:13.778472 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:30:13.778472 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:30:13.778472 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:30:13.778472 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:30:13.778472 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 14:30:13.778472 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:13.782257 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 258/517] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:30:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:13.782928 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5b73c-d9b1-4504-892a-a1f900209493 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:13.818438 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:13.882526 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48482287-c96c-420a-827c-cea4896666cc None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:13.888827 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5b73c-d9b1-4504-892a-a1f900209493 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:13.919874 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e449e-f20d-4b76-b57f-2b5c04256c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:13.920648 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e449e-f20d-4b76-b57f-2b5c04256c53 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:13.920947 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e449e-f20d-4b76-b57f-2b5c04256c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:13.921285 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e449e-f20d-4b76-b57f-2b5c04256c53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:13.923583 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae7c42-222c-4f54-ae93-336e78dda126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:13.924206 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae7c42-222c-4f54-ae93-336e78dda126 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:13.924488 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae7c42-222c-4f54-ae93-336e78dda126 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:13.924765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae7c42-222c-4f54-ae93-336e78dda126 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:13.925462 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae7c42-222c-4f54-ae93-336e78dda126 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:13.925768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae7c42-222c-4f54-ae93-336e78dda126 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:13.927171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae7c42-222c-4f54-ae93-336e78dda126 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:13.933300 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e449e-f20d-4b76-b57f-2b5c04256c53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:13.934839 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e449e-f20d-4b76-b57f-2b5c04256c53 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:13.944780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 259/518] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:13.944780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae7c42-222c-4f54-ae93-336e78dda126 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:13.956348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-695e449e-f20d-4b76-b57f-2b5c04256c53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:13.956348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:13.956348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:13.956348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:13.956348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:13.956348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:13.956348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:13.956348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:13.956348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:13.956348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:13.956348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:13.956348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:13.956348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:13.956348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:13.956348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:13.957408 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:13.957408 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:13.957408 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:13.957408 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:13.957408 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:13.957408 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:13.957408 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:13.957408 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:13.957408 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:13.957408 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:13.957408 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:13.957408 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:13.957408 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:13.957408 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:13.957408 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:13.957408 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:13.957408 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:13.958450 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:13.958450 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:13.958450 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:13.958450 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:13.964049 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 260/519] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:13.964049 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e449e-f20d-4b76-b57f-2b5c04256c53 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:14.010040 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:14.060739 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:14.067158 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae7c42-222c-4f54-ae93-336e78dda126 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:14.082857 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e449e-f20d-4b76-b57f-2b5c04256c53 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:14.104607 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81ad06-81d4-4f8b-bbfe-159dead45110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:14.105272 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81ad06-81d4-4f8b-bbfe-159dead45110 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:14.105971 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81ad06-81d4-4f8b-bbfe-159dead45110 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:14.106248 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81ad06-81d4-4f8b-bbfe-159dead45110 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:14.113652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b81ad06-81d4-4f8b-bbfe-159dead45110 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:14.115427 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b81ad06-81d4-4f8b-bbfe-159dead45110 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:14.120579 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d290f2-c988-4ffe-ae79-82c5bf96bde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:14.121292 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d290f2-c988-4ffe-ae79-82c5bf96bde5 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:14.121671 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d290f2-c988-4ffe-ae79-82c5bf96bde5 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:14.122041 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d290f2-c988-4ffe-ae79-82c5bf96bde5 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:14.122922 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d290f2-c988-4ffe-ae79-82c5bf96bde5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:14.123337 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d290f2-c988-4ffe-ae79-82c5bf96bde5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:14.124871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d290f2-c988-4ffe-ae79-82c5bf96bde5 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:14.128832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-8b81ad06-81d4-4f8b-bbfe-159dead45110 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:14.128832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:14.128832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:14.128832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:14.128832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:14.128832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:14.128832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:14.128832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:14.128832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:14.128832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:14.128832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:14.128832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:14.128832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:30:14.128832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:30:14.128832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:30:14.129793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:30:14.129793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:14.129793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:14.129793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:14.129793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:14.129793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:14.129793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:14.129793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:14.129793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:14.129793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:14.129793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:14.129793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:14.129793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:14.129793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:14.129793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:14.129793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:14.129793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:14.141470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:30:14.141470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:30:14.141470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:30:14.141470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:30:14.141470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:30:14.141470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:30:14.141470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:14.141470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:14.141470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 260/520] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:30:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:14.141470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81ad06-81d4-4f8b-bbfe-159dead45110 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:14.149821 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 261/521] 10.222.0.97 () {56 vars in 1137 bytes} [Tue Jul 26 14:30:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:14.149821 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d290f2-c988-4ffe-ae79-82c5bf96bde5 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:14.245936 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81ad06-81d4-4f8b-bbfe-159dead45110 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:14.246402 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:14.252319 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d290f2-c988-4ffe-ae79-82c5bf96bde5 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:14.287280 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1495fd-3c4a-42c4-9af4-78aead11c6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:14.287921 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1495fd-3c4a-42c4-9af4-78aead11c6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:14.288203 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1495fd-3c4a-42c4-9af4-78aead11c6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:14.288476 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1495fd-3c4a-42c4-9af4-78aead11c6c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:14.289161 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1495fd-3c4a-42c4-9af4-78aead11c6c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:14.289463 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1495fd-3c4a-42c4-9af4-78aead11c6c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:14.289670 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b9d53-12ef-46c4-8bf9-f05743cb0130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:14.290406 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b9d53-12ef-46c4-8bf9-f05743cb0130 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:14.291320 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b9d53-12ef-46c4-8bf9-f05743cb0130 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:14.291442 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1495fd-3c4a-42c4-9af4-78aead11c6c2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:14.291813 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b9d53-12ef-46c4-8bf9-f05743cb0130 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:14.300632 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30b9d53-12ef-46c4-8bf9-f05743cb0130 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:14.302191 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30b9d53-12ef-46c4-8bf9-f05743cb0130 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:14.302849 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 262/522] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:14.303727 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1495fd-3c4a-42c4-9af4-78aead11c6c2 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:14.319048 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-d30b9d53-12ef-46c4-8bf9-f05743cb0130 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:14.319048 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:14.319048 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:14.319048 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:14.319048 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:14.319048 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:14.319048 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:14.319048 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:14.319048 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:14.319048 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:14.319048 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:14.319048 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:14.319048 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:30:14.319048 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:30:14.319048 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:30:14.320042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:30:14.320042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:14.320042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:14.320042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:14.320042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:14.320042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:14.320042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:14.320042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:14.320042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:14.320042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:14.320042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:14.320042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:14.320042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:14.320042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:14.320042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:14.320042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:14.320042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:14.325874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:30:14.325874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:30:14.325874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:30:14.325874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:30:14.325874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:30:14.325874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:30:14.325874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:14.325874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:14.325874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 261/523] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:30:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:14.325874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b9d53-12ef-46c4-8bf9-f05743cb0130 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:14.341954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:14.420482 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1495fd-3c4a-42c4-9af4-78aead11c6c2 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:14.441513 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b9d53-12ef-46c4-8bf9-f05743cb0130 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:14.456587 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c84679-a888-4391-8837-3b7e37cd707c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:14.457212 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c84679-a888-4391-8837-3b7e37cd707c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:14.457501 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c84679-a888-4391-8837-3b7e37cd707c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:14.457780 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c84679-a888-4391-8837-3b7e37cd707c None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:14.458533 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c84679-a888-4391-8837-3b7e37cd707c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:14.458843 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c84679-a888-4391-8837-3b7e37cd707c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:14.460356 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c84679-a888-4391-8837-3b7e37cd707c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:14.483578 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 263/524] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:30:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:14.483578 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c84679-a888-4391-8837-3b7e37cd707c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:14.484767 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e1d51-d0b8-4d56-8dc0-22972a27b998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:14.485471 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e1d51-d0b8-4d56-8dc0-22972a27b998 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:14.485833 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e1d51-d0b8-4d56-8dc0-22972a27b998 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:14.486188 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e1d51-d0b8-4d56-8dc0-22972a27b998 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:14.503165 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:14.561026 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6e1d51-d0b8-4d56-8dc0-22972a27b998 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:14.562585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6e1d51-d0b8-4d56-8dc0-22972a27b998 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:14.577633 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:14.589099 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c84679-a888-4391-8837-3b7e37cd707c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:14.626769 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d88c93-99ee-48ff-9141-daa716657bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:14.627639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d88c93-99ee-48ff-9141-daa716657bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:14.627967 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d88c93-99ee-48ff-9141-daa716657bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:14.628249 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d88c93-99ee-48ff-9141-daa716657bd8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:14.635723 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d88c93-99ee-48ff-9141-daa716657bd8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:14.637165 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d88c93-99ee-48ff-9141-daa716657bd8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:14.649108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-73d88c93-99ee-48ff-9141-daa716657bd8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 14:30:14.649108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:14.649108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:14.649108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:14.649108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:14.649108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:14.649108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:14.649108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:14.649108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:14.649108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:14.649108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:14.649108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:14.649108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:30:14.649108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:30:14.649108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:30:14.649108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:30:14.650052 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:14.650052 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:14.650052 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:30:14.650052 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:14.650052 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:30:14.650052 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:14.650052 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:14.650052 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:14.650052 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:14.650052 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:14.650052 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:14.650052 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:14.650052 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:14.650052 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:14.650052 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:14.650052 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:14.650052 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:14.650052 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:14.651551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:14.651551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:14.651551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:30:14.651551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:30:14.651551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:30:14.651551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:30:14.651551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:30:14.651551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:30:14.651551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 14:30:14.651551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:14.655580 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 264/525] 10.222.0.97 () {56 vars in 1102 bytes} [Tue Jul 26 14:30:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:14.656288 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d88c93-99ee-48ff-9141-daa716657bd8 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:14.675095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 262/526] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:30:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:30:14.675095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e1d51-d0b8-4d56-8dc0-22972a27b998 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:14.744855 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:14.768312 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d88c93-99ee-48ff-9141-daa716657bd8 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:14.787165 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e1d51-d0b8-4d56-8dc0-22972a27b998 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:14.802062 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:14.808933 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d541f3-e5f6-42dd-ba28-3d448727c81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:14.809753 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d541f3-e5f6-42dd-ba28-3d448727c81b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:14.810209 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d541f3-e5f6-42dd-ba28-3d448727c81b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:14.810638 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d541f3-e5f6-42dd-ba28-3d448727c81b None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:14.811541 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d541f3-e5f6-42dd-ba28-3d448727c81b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:14.811858 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d541f3-e5f6-42dd-ba28-3d448727c81b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:14.813362 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d541f3-e5f6-42dd-ba28-3d448727c81b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:14.822831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6869b02-7105-473a-a54e-c3f5630008f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:14.823551 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6869b02-7105-473a-a54e-c3f5630008f9 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:14.823841 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6869b02-7105-473a-a54e-c3f5630008f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:14.824119 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6869b02-7105-473a-a54e-c3f5630008f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:14.824919 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6869b02-7105-473a-a54e-c3f5630008f9 None admin] RBAC: Adding query filter params (role.id=9074ca9226f8483c885f32fa868a1acc, scope.project.id=c4c8da3011294783b2180ade1bfd9366, user.id=266abef6cda5495b897170f92043aa03) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:14.825237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6869b02-7105-473a-a54e-c3f5630008f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:14.826672 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6869b02-7105-473a-a54e-c3f5630008f9 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:14.830766 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 265/527] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:30:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:14.831813 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d541f3-e5f6-42dd-ba28-3d448727c81b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:14.835437 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 263/528] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:30:14 2022] GET /identity/v3/role_assignments?user.id=266abef6cda5495b897170f92043aa03&scope.project.id=c4c8da3011294783b2180ade1bfd9366&role.id=9074ca9226f8483c885f32fa868a1acc => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:14.835437 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6869b02-7105-473a-a54e-c3f5630008f9 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:14.960285 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6869b02-7105-473a-a54e-c3f5630008f9 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:14.964369 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d541f3-e5f6-42dd-ba28-3d448727c81b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:15.001424 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6836af8-1ef1-48f7-9ffa-8cd7329cc54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:15.002074 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6836af8-1ef1-48f7-9ffa-8cd7329cc54a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:15.002359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6836af8-1ef1-48f7-9ffa-8cd7329cc54a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:15.002800 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6836af8-1ef1-48f7-9ffa-8cd7329cc54a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:15.011068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d9c4b-6e94-4d62-8d31-8cce8b07c0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:15.011740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d9c4b-6e94-4d62-8d31-8cce8b07c0fb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:15.012019 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d9c4b-6e94-4d62-8d31-8cce8b07c0fb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:15.012304 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d9c4b-6e94-4d62-8d31-8cce8b07c0fb None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:15.013151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72d9c4b-6e94-4d62-8d31-8cce8b07c0fb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:15.013416 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72d9c4b-6e94-4d62-8d31-8cce8b07c0fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:15.014891 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72d9c4b-6e94-4d62-8d31-8cce8b07c0fb None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:15.017011 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6836af8-1ef1-48f7-9ffa-8cd7329cc54a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:15.018783 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6836af8-1ef1-48f7-9ffa-8cd7329cc54a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:15.023723 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:15.035642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-c6836af8-1ef1-48f7-9ffa-8cd7329cc54a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:15.035642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:15.035642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:15.035642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:15.035642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:15.035642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:15.035642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:15.035642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:15.035642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:15.035642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:15.035642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:15.035642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:15.035642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:15.035642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:15.035642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:15.036525 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:15.036525 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:15.036525 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:15.036525 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:15.036525 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:15.036525 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:15.036525 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:15.036525 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:15.036525 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:15.036525 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:15.036525 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:15.036525 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:15.036525 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:15.036525 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:15.036525 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:15.036525 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:15.036525 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:15.037452 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:15.037452 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:15.037452 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:15.037452 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:15.041918 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 266/530] 10.222.0.97 () {56 vars in 1139 bytes} [Tue Jul 26 14:30:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:15.043442 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 264/530] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:15.043536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d9c4b-6e94-4d62-8d31-8cce8b07c0fb None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:15.043648 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6836af8-1ef1-48f7-9ffa-8cd7329cc54a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:15.088735 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:15.160150 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d9c4b-6e94-4d62-8d31-8cce8b07c0fb None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:15.192319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6836af8-1ef1-48f7-9ffa-8cd7329cc54a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:15.211712 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7844b9-7529-4ab1-b05a-8ad6a2848e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:15.213223 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7844b9-7529-4ab1-b05a-8ad6a2848e7d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:15.213743 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7844b9-7529-4ab1-b05a-8ad6a2848e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:15.214796 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7844b9-7529-4ab1-b05a-8ad6a2848e7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:15.216199 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7844b9-7529-4ab1-b05a-8ad6a2848e7d None admin] RBAC: Adding query filter params (role.id=9074ca9226f8483c885f32fa868a1acc, scope.project.id=c4c8da3011294783b2180ade1bfd9366, user.id=62adb1f5e9654927847e1717630ff6b9) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:15.216704 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7844b9-7529-4ab1-b05a-8ad6a2848e7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:15.219278 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7844b9-7529-4ab1-b05a-8ad6a2848e7d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:15.227842 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 267/531] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:30:15 2022] GET /identity/v3/role_assignments?user.id=62adb1f5e9654927847e1717630ff6b9&scope.project.id=c4c8da3011294783b2180ade1bfd9366&role.id=9074ca9226f8483c885f32fa868a1acc => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:15.229025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7844b9-7529-4ab1-b05a-8ad6a2848e7d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:15.237460 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff678e-658b-4f0e-802b-b3cbe654d60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:15.238191 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff678e-658b-4f0e-802b-b3cbe654d60c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:15.238551 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff678e-658b-4f0e-802b-b3cbe654d60c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:15.238904 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff678e-658b-4f0e-802b-b3cbe654d60c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:15.325240 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:15.350914 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ff678e-658b-4f0e-802b-b3cbe654d60c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:15.352448 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ff678e-658b-4f0e-802b-b3cbe654d60c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:15.382455 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7844b9-7529-4ab1-b05a-8ad6a2848e7d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:15.387135 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:15.428858 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b767b-7efb-4265-9892-cd2546dcb99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:15.429588 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b767b-7efb-4265-9892-cd2546dcb99d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:15.429955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b767b-7efb-4265-9892-cd2546dcb99d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:15.430302 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b767b-7efb-4265-9892-cd2546dcb99d None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:15.431089 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8b767b-7efb-4265-9892-cd2546dcb99d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:15.431560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8b767b-7efb-4265-9892-cd2546dcb99d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:15.433016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8b767b-7efb-4265-9892-cd2546dcb99d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:15.448313 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 268/532] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:15.449904 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814bb7b0-015b-4ef2-908f-b6d91b61173f None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:15.450318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814bb7b0-015b-4ef2-908f-b6d91b61173f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:15.450717 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814bb7b0-015b-4ef2-908f-b6d91b61173f None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:15.452115 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 269/533] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:15.457006 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814bb7b0-015b-4ef2-908f-b6d91b61173f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:15.473238 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 265/534] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:30:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:30:15.474639 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5364a5-31fc-40b4-ac29-ba9883b39e9a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:15.475107 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5364a5-31fc-40b4-ac29-ba9883b39e9a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:15.475503 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5364a5-31fc-40b4-ac29-ba9883b39e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:15.495392 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ca5364a5-31fc-40b4-ac29-ba9883b39e9a None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:15.497067 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ca5364a5-31fc-40b4-ac29-ba9883b39e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:15.574728 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 266/535] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:30:15.575735 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5364a5-31fc-40b4-ac29-ba9883b39e9a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:15.596671 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814bb7b0-015b-4ef2-908f-b6d91b61173f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:15.617030 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:15.637414 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4047fb4-d131-42bb-acd9-077b9f16f2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:15.638151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4047fb4-d131-42bb-acd9-077b9f16f2f3 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:15.638497 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4047fb4-d131-42bb-acd9-077b9f16f2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:15.638838 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4047fb4-d131-42bb-acd9-077b9f16f2f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:15.645434 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:15.646904 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4047fb4-d131-42bb-acd9-077b9f16f2f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:15.648438 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4047fb4-d131-42bb-acd9-077b9f16f2f3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:15.662429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-e4047fb4-d131-42bb-acd9-077b9f16f2f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:15.662429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:15.662429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:15.662429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:15.662429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:15.662429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:15.662429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:15.662429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:15.662429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:15.662429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:15.662429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:15.662429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:15.662429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:30:15.662429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:30:15.662429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:30:15.663377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:30:15.663377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:15.663377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:15.663377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:15.663377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:15.663377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:15.663377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:15.663377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:15.663377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:15.663377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:15.663377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:15.663377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:15.663377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:15.663377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:15.663377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:15.663377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:15.663377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:15.664655 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:30:15.664655 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:30:15.664655 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:30:15.664655 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:30:15.664655 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:30:15.664655 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:30:15.664655 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:15.664655 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:15.664655 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 270/536] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:30:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:15.664655 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4047fb4-d131-42bb-acd9-077b9f16f2f3 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:15.691080 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5364a5-31fc-40b4-ac29-ba9883b39e9a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:15.730044 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d088f68d-226b-4c98-9765-d23758744a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:15.730639 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d088f68d-226b-4c98-9765-d23758744a57 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:15.730930 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d088f68d-226b-4c98-9765-d23758744a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:15.731207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d088f68d-226b-4c98-9765-d23758744a57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:15.742820 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d088f68d-226b-4c98-9765-d23758744a57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:15.744643 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d088f68d-226b-4c98-9765-d23758744a57 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:15.761359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-d088f68d-226b-4c98-9765-d23758744a57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:15.761359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:15.761359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:15.761359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:15.761359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:15.761359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:15.761359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:15.761359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:15.761359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:15.761359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:15.761359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:15.761359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:15.761359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:15.761359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:15.761359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:15.763450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:15.763450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:15.763450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:15.763450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:15.763450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:15.763450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:15.763450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:15.763450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:15.763450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:15.763450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:15.763450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:15.763450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:15.763450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:15.763450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:15.763450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:15.763450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:15.763450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:15.765566 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:15.765566 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:15.765566 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:15.765566 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:15.765566 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 267/537] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:15.768588 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d088f68d-226b-4c98-9765-d23758744a57 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:15.779803 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4047fb4-d131-42bb-acd9-077b9f16f2f3 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:15.816552 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b02b43-054d-41e3-babc-c6d468e2da74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:15.817341 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b02b43-054d-41e3-babc-c6d468e2da74 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:15.817744 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b02b43-054d-41e3-babc-c6d468e2da74 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:15.818049 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b02b43-054d-41e3-babc-c6d468e2da74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:15.832283 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b02b43-054d-41e3-babc-c6d468e2da74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:15.840407 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-93b02b43-054d-41e3-babc-c6d468e2da74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:15.840407 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:15.840407 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:15.840407 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:15.840407 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:15.840407 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:15.840407 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:15.840407 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:15.840407 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:15.840407 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:15.840407 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:15.840407 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:15.840407 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:15.840407 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:15.840407 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:15.840407 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:15.842547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:15.842547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:15.842547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:15.842547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:15.842547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:15.842547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:15.842547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:15.842547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:15.842547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:15.842547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:15.842547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:15.842547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:15.842547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:15.842547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:15.842547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:15.842547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:15.844636 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:15.844636 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:15.844636 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:15.844636 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:15.844636 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:15.844636 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:15.844636 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:15.844636 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:15.844636 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:15.844636 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:15.844636 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:15.844636 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:15.844636 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 271/538] 10.222.0.97 () {56 vars in 1102 bytes} [Tue Jul 26 14:30:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:15.844636 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b02b43-054d-41e3-babc-c6d468e2da74 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:15.872734 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d088f68d-226b-4c98-9765-d23758744a57 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:15.886603 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:15.887489 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:15.910784 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425d056-27c8-45af-87c9-b7617a1db830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:15.911689 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425d056-27c8-45af-87c9-b7617a1db830 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:15.912120 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425d056-27c8-45af-87c9-b7617a1db830 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:15.912572 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425d056-27c8-45af-87c9-b7617a1db830 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:15.913525 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8425d056-27c8-45af-87c9-b7617a1db830 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:15.913980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8425d056-27c8-45af-87c9-b7617a1db830 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:15.916322 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8425d056-27c8-45af-87c9-b7617a1db830 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:15.936453 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 268/539] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:30:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:15.937628 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425d056-27c8-45af-87c9-b7617a1db830 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:15.960807 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b02b43-054d-41e3-babc-c6d468e2da74 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:16.005709 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6387b8-cdf8-4e04-98db-4866a25ce0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:16.006419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6387b8-cdf8-4e04-98db-4866a25ce0b0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:16.006700 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6387b8-cdf8-4e04-98db-4866a25ce0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:16.006975 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6387b8-cdf8-4e04-98db-4866a25ce0b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:16.010278 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6387b8-cdf8-4e04-98db-4866a25ce0b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:16.010708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6387b8-cdf8-4e04-98db-4866a25ce0b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:16.012502 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6387b8-cdf8-4e04-98db-4866a25ce0b0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:16.035438 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 272/540] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:16.035438 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6387b8-cdf8-4e04-98db-4866a25ce0b0 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:16.072108 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425d056-27c8-45af-87c9-b7617a1db830 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:16.111259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83310c0d-3890-4e78-b390-60f089111b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:16.112164 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83310c0d-3890-4e78-b390-60f089111b01 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:16.112650 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83310c0d-3890-4e78-b390-60f089111b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:16.113101 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83310c0d-3890-4e78-b390-60f089111b01 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:16.114082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83310c0d-3890-4e78-b390-60f089111b01 None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:16.114568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83310c0d-3890-4e78-b390-60f089111b01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:16.116242 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83310c0d-3890-4e78-b390-60f089111b01 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:16.125985 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 269/541] 10.222.0.97 () {56 vars in 1103 bytes} [Tue Jul 26 14:30:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:16.127269 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83310c0d-3890-4e78-b390-60f089111b01 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:16.143280 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:16.148116 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6387b8-cdf8-4e04-98db-4866a25ce0b0 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:16.167999 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:16.190605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce680b94-11e8-4aea-8a7d-1c610bfbbd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:16.191866 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce680b94-11e8-4aea-8a7d-1c610bfbbd65 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:16.192203 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce680b94-11e8-4aea-8a7d-1c610bfbbd65 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:16.192535 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce680b94-11e8-4aea-8a7d-1c610bfbbd65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:16.193530 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce680b94-11e8-4aea-8a7d-1c610bfbbd65 None admin] RBAC: Adding query filter params (role.id=9074ca9226f8483c885f32fa868a1acc, scope.project.id=c4c8da3011294783b2180ade1bfd9366, user.id=bad45b307edc415fa6185a82af64ede4) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:16.193940 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce680b94-11e8-4aea-8a7d-1c610bfbbd65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:16.195492 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce680b94-11e8-4aea-8a7d-1c610bfbbd65 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:16.206162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 273/542] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:30:16 2022] GET /identity/v3/role_assignments?user.id=bad45b307edc415fa6185a82af64ede4&scope.project.id=c4c8da3011294783b2180ade1bfd9366&role.id=9074ca9226f8483c885f32fa868a1acc => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:16.207255 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce680b94-11e8-4aea-8a7d-1c610bfbbd65 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:16.242901 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83310c0d-3890-4e78-b390-60f089111b01 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:16.302189 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a296f9-18cb-468a-8803-b270df7168b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:16.303279 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a296f9-18cb-468a-8803-b270df7168b8 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:16.303752 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a296f9-18cb-468a-8803-b270df7168b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:16.304234 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a296f9-18cb-468a-8803-b270df7168b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:16.317521 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a296f9-18cb-468a-8803-b270df7168b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:16.320930 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a296f9-18cb-468a-8803-b270df7168b8 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:16.344689 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-40a296f9-18cb-468a-8803-b270df7168b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:16.344689 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:16.344689 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:16.344689 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:16.344689 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:16.344689 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:16.344689 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:16.344689 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:16.344689 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:16.344689 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:16.344689 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:16.344689 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:16.344689 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:30:16.344689 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:30:16.344689 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:30:16.345819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:30:16.345819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:16.345819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:16.345819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:16.345819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:16.345819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:16.345819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:16.345819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:16.345819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:16.345819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:16.345819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:16.345819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:16.345819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:16.345819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:16.345819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:16.345819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:16.345819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:16.346866 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:30:16.346866 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:30:16.346866 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:30:16.346866 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:30:16.346866 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:30:16.346866 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:30:16.346866 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:16.346866 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:16.355349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 270/543] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:30:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:16.369120 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a296f9-18cb-468a-8803-b270df7168b8 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:16.434178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce680b94-11e8-4aea-8a7d-1c610bfbbd65 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:16.490642 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056e10e-8a27-4599-9533-0fcbff86f2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:16.491321 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056e10e-8a27-4599-9533-0fcbff86f2d7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:16.491685 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056e10e-8a27-4599-9533-0fcbff86f2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:16.491970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056e10e-8a27-4599-9533-0fcbff86f2d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:16.512006 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:16.517520 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9056e10e-8a27-4599-9533-0fcbff86f2d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:16.518851 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9056e10e-8a27-4599-9533-0fcbff86f2d7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:16.524696 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a296f9-18cb-468a-8803-b270df7168b8 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:16.554928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-9056e10e-8a27-4599-9533-0fcbff86f2d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:16.554928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:16.554928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:16.554928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:16.554928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:16.554928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:16.554928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:16.554928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:16.554928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:16.554928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:16.554928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:16.554928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:16.554928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:16.554928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:16.554928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:16.557925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:16.557925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:16.557925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:16.557925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:16.557925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:16.557925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:16.557925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:16.557925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:16.557925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:16.557925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:16.557925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:16.557925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:16.557925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:16.557925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:16.557925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:16.557925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:16.557925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:16.558743 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:16.558743 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:16.558743 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:16.558743 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:16.559996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 274/544] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:16.568560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056e10e-8a27-4599-9533-0fcbff86f2d7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:16.600201 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:16.610283 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90fec1f-dbf2-4294-a953-7496b39da6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:16.611268 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90fec1f-dbf2-4294-a953-7496b39da6eb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:16.611764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90fec1f-dbf2-4294-a953-7496b39da6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:16.612220 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90fec1f-dbf2-4294-a953-7496b39da6eb None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:16.613157 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90fec1f-dbf2-4294-a953-7496b39da6eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:16.613665 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90fec1f-dbf2-4294-a953-7496b39da6eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:16.615274 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90fec1f-dbf2-4294-a953-7496b39da6eb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:16.629837 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 271/545] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:30:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:30:16.636101 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90fec1f-dbf2-4294-a953-7496b39da6eb None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:16.781158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056e10e-8a27-4599-9533-0fcbff86f2d7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:16.831258 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90fec1f-dbf2-4294-a953-7496b39da6eb None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:16.833333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4582f7d7-ca94-494d-8c95-8680d6981df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:16.834391 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4582f7d7-ca94-494d-8c95-8680d6981df9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:16.834748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4582f7d7-ca94-494d-8c95-8680d6981df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:16.835036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4582f7d7-ca94-494d-8c95-8680d6981df9 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:16.835798 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4582f7d7-ca94-494d-8c95-8680d6981df9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:16.836143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4582f7d7-ca94-494d-8c95-8680d6981df9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:16.838087 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4582f7d7-ca94-494d-8c95-8680d6981df9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:16.856922 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 275/546] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:16.858421 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4582f7d7-ca94-494d-8c95-8680d6981df9 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:16.917677 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b2296-a67f-4fde-bde6-d2a360920172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:16.918311 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b2296-a67f-4fde-bde6-d2a360920172 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:16.918602 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b2296-a67f-4fde-bde6-d2a360920172 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:16.919331 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b2296-a67f-4fde-bde6-d2a360920172 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:16.919727 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b2296-a67f-4fde-bde6-d2a360920172 None admin] RBAC: Adding query filter params (role.id=9074ca9226f8483c885f32fa868a1acc, scope.project.id=c4c8da3011294783b2180ade1bfd9366, user.id=c81180fb4ff04c1186a21e43fb597a91) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:16.919997 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b2296-a67f-4fde-bde6-d2a360920172 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:16.921439 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b2296-a67f-4fde-bde6-d2a360920172 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:16.934893 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 272/547] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:30:16 2022] GET /identity/v3/role_assignments?user.id=c81180fb4ff04c1186a21e43fb597a91&scope.project.id=c4c8da3011294783b2180ade1bfd9366&role.id=9074ca9226f8483c885f32fa868a1acc => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:16.946977 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:16.975120 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4582f7d7-ca94-494d-8c95-8680d6981df9 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:17.013587 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c17bb-f45f-427d-b172-9b2df08caf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:17.014283 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c17bb-f45f-427d-b172-9b2df08caf7f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:17.014569 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c17bb-f45f-427d-b172-9b2df08caf7f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:17.014849 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c17bb-f45f-427d-b172-9b2df08caf7f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:17.021389 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0c6dc-207b-4d9c-bc2f-e8ca72ba2322 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:17.021757 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0c6dc-207b-4d9c-bc2f-e8ca72ba2322 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:17.022012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0c6dc-207b-4d9c-bc2f-e8ca72ba2322 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:17.023576 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 273/548] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:17.023756 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c17bb-f45f-427d-b172-9b2df08caf7f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:17.025323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c17bb-f45f-427d-b172-9b2df08caf7f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:17.029294 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c0734-c811-4527-b762-995329c38c3f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:17.029572 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c0734-c811-4527-b762-995329c38c3f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:17.029845 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c0734-c811-4527-b762-995329c38c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:17.040063 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-984c17bb-f45f-427d-b172-9b2df08caf7f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 14:30:17.040063 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:17.040063 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:17.040063 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:17.040063 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:17.040063 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:17.040063 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:17.040063 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:17.040063 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:17.040063 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:17.040063 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:17.040063 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:17.040063 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:30:17.040063 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:30:17.040063 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:30:17.040063 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:30:17.043000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:17.043000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:17.043000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:30:17.043000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:17.043000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:30:17.043000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:17.043000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:17.043000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:17.043000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:17.043000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:17.043000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:17.043000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:17.043000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:17.043000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:17.043000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:17.043000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:17.043000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:17.043000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:17.047343 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:17.047343 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:17.047343 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:30:17.047343 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:30:17.047343 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:30:17.047343 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:30:17.047343 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:30:17.047343 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:30:17.047343 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 14:30:17.047343 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:17.047343 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 276/549] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:30:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:30:17.049177 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c17bb-f45f-427d-b172-9b2df08caf7f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:17.085243 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-184c0734-c811-4527-b762-995329c38c3f None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:17.087202 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-184c0734-c811-4527-b762-995329c38c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:17.121300 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:17.168116 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c17bb-f45f-427d-b172-9b2df08caf7f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:17.196542 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 274/550] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:30:17.208155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19cb0de-76a2-4df3-8c40-8b978150ae0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:17.208785 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19cb0de-76a2-4df3-8c40-8b978150ae0c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:17.209070 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19cb0de-76a2-4df3-8c40-8b978150ae0c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:17.209349 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19cb0de-76a2-4df3-8c40-8b978150ae0c None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:17.210155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19cb0de-76a2-4df3-8c40-8b978150ae0c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:17.210467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19cb0de-76a2-4df3-8c40-8b978150ae0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:17.211961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19cb0de-76a2-4df3-8c40-8b978150ae0c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:17.230751 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c0734-c811-4527-b762-995329c38c3f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:17.235475 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 277/551] 10.222.0.97 () {56 vars in 1135 bytes} [Tue Jul 26 14:30:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:30:17.235974 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:17.236924 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19cb0de-76a2-4df3-8c40-8b978150ae0c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:17.415155 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c0734-c811-4527-b762-995329c38c3f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:17.419651 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19cb0de-76a2-4df3-8c40-8b978150ae0c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:17.467459 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca5df0d-a3fe-4e27-8f7b-e5709d2d4f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:17.468110 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5df0d-a3fe-4e27-8f7b-e5709d2d4f4d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:17.468395 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5df0d-a3fe-4e27-8f7b-e5709d2d4f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:17.468671 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5df0d-a3fe-4e27-8f7b-e5709d2d4f4d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:17.481227 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca5df0d-a3fe-4e27-8f7b-e5709d2d4f4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:17.486555 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-4ca5df0d-a3fe-4e27-8f7b-e5709d2d4f4d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:17.486555 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:17.486555 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:17.486555 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:17.486555 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:17.486555 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:17.486555 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:17.486555 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:17.486555 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:17.486555 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:17.486555 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:17.486555 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:17.486555 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:17.486555 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:17.486555 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:17.486555 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:17.487682 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:17.487682 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:17.487682 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:17.487682 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:17.487682 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:17.487682 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:17.487682 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:17.487682 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:17.487682 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:17.487682 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:17.487682 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:17.487682 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:17.487682 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:17.487682 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:17.487682 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:17.487682 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:17.488708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:17.488708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:17.488708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:17.488708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:17.488708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:17.488708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:17.488708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:17.488708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:17.488708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:17.488708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:17.488708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:17.488708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:17.488708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 275/552] 10.222.0.97 () {56 vars in 1102 bytes} [Tue Jul 26 14:30:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:30:17.493465 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca5df0d-a3fe-4e27-8f7b-e5709d2d4f4d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:17.506000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c8048-88fd-4edd-b3ab-b418946142bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:17.506727 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c8048-88fd-4edd-b3ab-b418946142bd None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:17.507079 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c8048-88fd-4edd-b3ab-b418946142bd None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:17.507467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c8048-88fd-4edd-b3ab-b418946142bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:17.513777 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:17.626889 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c8048-88fd-4edd-b3ab-b418946142bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:17.628421 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c8048-88fd-4edd-b3ab-b418946142bd None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:17.635750 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:17.681051 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca5df0d-a3fe-4e27-8f7b-e5709d2d4f4d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:17.719856 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f23e53-70d4-49d0-a5b0-b5bbdf3e7481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:17.720571 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f23e53-70d4-49d0-a5b0-b5bbdf3e7481 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:17.720858 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f23e53-70d4-49d0-a5b0-b5bbdf3e7481 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:17.721131 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f23e53-70d4-49d0-a5b0-b5bbdf3e7481 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:17.721844 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f23e53-70d4-49d0-a5b0-b5bbdf3e7481 None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:17.722252 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f23e53-70d4-49d0-a5b0-b5bbdf3e7481 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:17.723675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f23e53-70d4-49d0-a5b0-b5bbdf3e7481 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:17.733486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 276/553] 10.222.0.97 () {56 vars in 1103 bytes} [Tue Jul 26 14:30:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:17.734792 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52737bdc-d54e-439c-82f8-7f03f628231f None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:17.735175 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52737bdc-d54e-439c-82f8-7f03f628231f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:17.735484 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52737bdc-d54e-439c-82f8-7f03f628231f None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:17.737002 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 277/554] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:17.738398 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52737bdc-d54e-439c-82f8-7f03f628231f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:17.762194 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 278/555] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:30:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 14:30:17.763581 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3fe54-9ff9-413b-8307-462cc513220e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:17.763885 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3fe54-9ff9-413b-8307-462cc513220e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:17.764170 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3fe54-9ff9-413b-8307-462cc513220e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:17.782953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-40b3fe54-9ff9-413b-8307-462cc513220e None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:17.784246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-40b3fe54-9ff9-413b-8307-462cc513220e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:17.825863 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:17.867497 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 279/556] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:30:17.868395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3fe54-9ff9-413b-8307-462cc513220e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:17.869428 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52737bdc-d54e-439c-82f8-7f03f628231f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:17.902701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:17.920323 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a4497-5157-4b11-8c4c-e4ad0acd1ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:17.920996 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a4497-5157-4b11-8c4c-e4ad0acd1ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:17.921280 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a4497-5157-4b11-8c4c-e4ad0acd1ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:17.921558 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a4497-5157-4b11-8c4c-e4ad0acd1ee0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:17.935785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77a4497-5157-4b11-8c4c-e4ad0acd1ee0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:17.937933 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77a4497-5157-4b11-8c4c-e4ad0acd1ee0 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:17.957187 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-b77a4497-5157-4b11-8c4c-e4ad0acd1ee0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:17.957187 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:17.957187 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:17.957187 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:17.957187 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:17.957187 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:17.957187 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:17.957187 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:17.957187 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:17.957187 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:17.957187 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:17.957187 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:17.957187 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:17.957187 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:17.957187 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:17.958157 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:17.958157 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:17.958157 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:17.958157 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:17.958157 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:17.958157 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:17.958157 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:17.958157 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:17.958157 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:17.958157 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:17.958157 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:17.958157 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:17.958157 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:17.958157 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:17.958157 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:17.958157 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:17.958157 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:17.959671 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:17.959671 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:17.959671 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:17.959671 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:17.959671 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 278/557] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:17.959671 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a4497-5157-4b11-8c4c-e4ad0acd1ee0 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:17.994854 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3fe54-9ff9-413b-8307-462cc513220e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:18.031171 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8a257-6367-4cf5-a42d-e6a0e6e22a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:18.032008 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8a257-6367-4cf5-a42d-e6a0e6e22a6a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:18.032393 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8a257-6367-4cf5-a42d-e6a0e6e22a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:18.032769 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8a257-6367-4cf5-a42d-e6a0e6e22a6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:18.047835 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed8a257-6367-4cf5-a42d-e6a0e6e22a6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:18.051696 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed8a257-6367-4cf5-a42d-e6a0e6e22a6a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:18.069219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-bed8a257-6367-4cf5-a42d-e6a0e6e22a6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:18.069219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:18.069219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:18.069219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:18.069219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:18.069219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:18.069219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:18.069219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:18.069219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:18.069219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:18.069219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:18.069219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:18.069219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:18.069219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:18.069219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:18.070673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:18.070673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:18.070673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:18.070673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:18.070673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:18.070673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:18.070673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:18.070673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:18.070673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:18.070673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:18.070673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:18.070673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:18.070673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:18.070673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:18.070673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:18.070673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:18.070673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:18.071551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:18.071551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:18.071551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:18.071551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:18.071551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 280/558] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:18.071551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8a257-6367-4cf5-a42d-e6a0e6e22a6a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:18.106350 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a4497-5157-4b11-8c4c-e4ad0acd1ee0 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:18.140824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:18.154773 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4202eec-025d-42c4-b802-853151f32d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:18.155455 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4202eec-025d-42c4-b802-853151f32d75 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:18.155752 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4202eec-025d-42c4-b802-853151f32d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:18.156204 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4202eec-025d-42c4-b802-853151f32d75 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:18.163262 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:18.167166 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4202eec-025d-42c4-b802-853151f32d75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:18.172379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-d4202eec-025d-42c4-b802-853151f32d75 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:18.172379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:18.172379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:18.172379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:18.172379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:18.172379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:18.172379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:18.172379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:18.172379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:18.172379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:18.172379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:18.172379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:18.172379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:18.172379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:18.172379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:18.172379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:18.173297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:18.173297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:18.173297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:18.173297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:18.173297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:18.173297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:18.173297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:18.173297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:18.173297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:18.173297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:18.173297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:18.173297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:18.173297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:18.173297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:18.173297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:18.173297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:18.174170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:18.174170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:18.174170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:18.174170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:18.174170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:18.174170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:18.174170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:18.174170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:18.174170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:18.174170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:18.174170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:18.174170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:18.178727 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 279/559] 10.222.0.97 () {56 vars in 1102 bytes} [Tue Jul 26 14:30:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:18.179603 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4202eec-025d-42c4-b802-853151f32d75 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:18.193941 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8a257-6367-4cf5-a42d-e6a0e6e22a6a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:18.258661 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d967c5d-26a0-4a51-a3f0-5fa66bb2e62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:18.259324 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d967c5d-26a0-4a51-a3f0-5fa66bb2e62b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:18.259655 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d967c5d-26a0-4a51-a3f0-5fa66bb2e62b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:18.259939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d967c5d-26a0-4a51-a3f0-5fa66bb2e62b None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:18.260636 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d967c5d-26a0-4a51-a3f0-5fa66bb2e62b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:18.260947 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d967c5d-26a0-4a51-a3f0-5fa66bb2e62b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:18.262321 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d967c5d-26a0-4a51-a3f0-5fa66bb2e62b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:18.276552 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 281/560] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:18.277542 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d967c5d-26a0-4a51-a3f0-5fa66bb2e62b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:18.331433 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4202eec-025d-42c4-b802-853151f32d75 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:18.370178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ef3c9-3ff5-4e6e-be8c-dae87d029cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:18.371230 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ef3c9-3ff5-4e6e-be8c-dae87d029cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:18.371829 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ef3c9-3ff5-4e6e-be8c-dae87d029cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:18.372319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ef3c9-3ff5-4e6e-be8c-dae87d029cd4 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:18.373266 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ef3c9-3ff5-4e6e-be8c-dae87d029cd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:18.373808 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ef3c9-3ff5-4e6e-be8c-dae87d029cd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:18.375617 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ef3c9-3ff5-4e6e-be8c-dae87d029cd4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:18.387635 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d967c5d-26a0-4a51-a3f0-5fa66bb2e62b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:18.395132 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 280/561] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:18.396396 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ef3c9-3ff5-4e6e-be8c-dae87d029cd4 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:18.426381 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04359ad3-8134-4283-8c31-111e0badeed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:18.427104 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04359ad3-8134-4283-8c31-111e0badeed2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:18.427287 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04359ad3-8134-4283-8c31-111e0badeed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:18.427625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04359ad3-8134-4283-8c31-111e0badeed2 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:18.428343 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04359ad3-8134-4283-8c31-111e0badeed2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:18.428654 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04359ad3-8134-4283-8c31-111e0badeed2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:18.430136 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04359ad3-8134-4283-8c31-111e0badeed2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:18.432220 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:18.440897 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 282/562] 10.222.0.97 () {56 vars in 1103 bytes} [Tue Jul 26 14:30:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:18.445948 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04359ad3-8134-4283-8c31-111e0badeed2 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:18.454970 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:18.524876 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ef3c9-3ff5-4e6e-be8c-dae87d029cd4 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:18.563642 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04359ad3-8134-4283-8c31-111e0badeed2 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:18.565731 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b45b1-59b9-4e2a-a584-2afd42bbd48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:18.567736 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b45b1-59b9-4e2a-a584-2afd42bbd48c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:18.568026 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b45b1-59b9-4e2a-a584-2afd42bbd48c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:18.568414 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b45b1-59b9-4e2a-a584-2afd42bbd48c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:18.577329 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5b45b1-59b9-4e2a-a584-2afd42bbd48c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:18.578774 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5b45b1-59b9-4e2a-a584-2afd42bbd48c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:18.590447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-2a5b45b1-59b9-4e2a-a584-2afd42bbd48c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:30:18.590447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:18.590447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:18.590447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:18.590447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:18.590447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:18.590447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:18.590447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:18.590447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:18.590447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:18.590447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:18.590447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:18.590447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:30:18.590447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:30:18.590447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:30:18.590447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:30:18.591760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:18.591760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:18.591760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:30:18.591760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:18.591760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:30:18.591760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:18.591760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:18.591760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:18.591760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:18.591760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:18.591760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:18.591760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:18.591760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:18.591760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:18.591760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:18.591760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:18.591760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:18.591760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:18.593022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:18.593022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:18.593022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:30:18.593022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:30:18.593022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:30:18.593022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:30:18.593022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:30:18.593022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:30:18.593022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:30:18.593022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:18.593022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 281/563] 10.222.0.97 () {56 vars in 1093 bytes} [Tue Jul 26 14:30:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:18.593022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b45b1-59b9-4e2a-a584-2afd42bbd48c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:18.610769 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67055fc9-b9f9-447d-a958-a4675ca5931f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:18.611611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67055fc9-b9f9-447d-a958-a4675ca5931f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:18.611747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67055fc9-b9f9-447d-a958-a4675ca5931f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:18.611977 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67055fc9-b9f9-447d-a958-a4675ca5931f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:18.618928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67055fc9-b9f9-447d-a958-a4675ca5931f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:18.620383 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67055fc9-b9f9-447d-a958-a4675ca5931f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:18.632000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-67055fc9-b9f9-447d-a958-a4675ca5931f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:18.632000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:18.632000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:18.632000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:18.632000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:18.632000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:18.632000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:18.632000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:18.632000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:18.632000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:18.632000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:18.632000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:18.632000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:30:18.632000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:30:18.632000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:30:18.633437 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:30:18.633437 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:18.633437 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:18.633437 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:18.633437 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:18.633437 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:18.633437 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:18.633437 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:18.633437 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:18.633437 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:18.633437 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:18.633437 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:18.633437 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:18.633437 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:18.633437 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:18.633437 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:18.633437 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:18.636054 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:30:18.636054 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:30:18.636054 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:30:18.636054 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:30:18.636054 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:30:18.636054 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:30:18.636054 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:18.636054 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:18.636054 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 283/564] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:30:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:18.636054 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861183c-e125-4e04-afe9-aa39657bae67 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:18.636054 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861183c-e125-4e04-afe9-aa39657bae67 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:18.636655 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861183c-e125-4e04-afe9-aa39657bae67 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:18.640616 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 284/565] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:18.641675 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b861183c-e125-4e04-afe9-aa39657bae67 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:18.713447 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:18.722593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b45b1-59b9-4e2a-a584-2afd42bbd48c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:18.726227 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:18.763395 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0a7cf-3ce3-4718-9bee-4feef23d6c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:18.764216 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b861183c-e125-4e04-afe9-aa39657bae67 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:18.764510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0a7cf-3ce3-4718-9bee-4feef23d6c9d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:18.765002 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0a7cf-3ce3-4718-9bee-4feef23d6c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:18.765412 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0a7cf-3ce3-4718-9bee-4feef23d6c9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:18.778361 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef0a7cf-3ce3-4718-9bee-4feef23d6c9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:18.780258 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef0a7cf-3ce3-4718-9bee-4feef23d6c9d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:18.795892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-0ef0a7cf-3ce3-4718-9bee-4feef23d6c9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:18.795892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:18.795892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:18.795892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:18.795892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:18.795892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:18.795892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:18.795892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:18.795892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:18.795892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:18.795892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:18.795892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:18.795892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:18.795892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:18.795892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:18.796913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:18.796913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:18.796913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:18.796913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:18.796913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:18.796913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:18.796913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:18.796913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:18.796913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:18.796913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:18.796913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:18.796913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:18.796913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:18.796913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:18.796913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:18.796913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:18.796913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:18.797939 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:18.797939 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:18.797939 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:18.797939 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:18.799426 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 282/566] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:18.800361 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0a7cf-3ce3-4718-9bee-4feef23d6c9d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:18.808376 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa299a16-4fec-4b2d-859b-d82e924ef667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:18.809013 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa299a16-4fec-4b2d-859b-d82e924ef667 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:18.809298 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa299a16-4fec-4b2d-859b-d82e924ef667 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:18.809578 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa299a16-4fec-4b2d-859b-d82e924ef667 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:18.810381 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa299a16-4fec-4b2d-859b-d82e924ef667 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:18.810691 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa299a16-4fec-4b2d-859b-d82e924ef667 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:18.812188 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa299a16-4fec-4b2d-859b-d82e924ef667 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:18.831155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 285/567] 10.222.0.97 () {56 vars in 1133 bytes} [Tue Jul 26 14:30:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:18.832884 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14e3f5-21b2-4795-b059-22f05d999222 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:18.833236 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14e3f5-21b2-4795-b059-22f05d999222 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:18.833616 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14e3f5-21b2-4795-b059-22f05d999222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:18.853263 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9d14e3f5-21b2-4795-b059-22f05d999222 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:18.854597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9d14e3f5-21b2-4795-b059-22f05d999222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:18.921531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0a7cf-3ce3-4718-9bee-4feef23d6c9d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:18.927236 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 286/568] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:30:18.928185 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14e3f5-21b2-4795-b059-22f05d999222 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:18.965091 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ca7c3-cebf-4210-97b5-73cc5365b46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:18.965630 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ca7c3-cebf-4210-97b5-73cc5365b46c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:18.965917 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ca7c3-cebf-4210-97b5-73cc5365b46c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:18.966198 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ca7c3-cebf-4210-97b5-73cc5365b46c None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:18.966954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060ca7c3-cebf-4210-97b5-73cc5365b46c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:18.967269 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060ca7c3-cebf-4210-97b5-73cc5365b46c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:18.968764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060ca7c3-cebf-4210-97b5-73cc5365b46c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:18.980955 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 283/569] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:30:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:18.981763 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ca7c3-cebf-4210-97b5-73cc5365b46c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:18.990367 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:18.999934 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:19.060611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14e3f5-21b2-4795-b059-22f05d999222 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:19.090586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ca7c3-cebf-4210-97b5-73cc5365b46c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:19.099065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0486c4-368a-4389-84db-b56fda86816f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:19.099809 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0486c4-368a-4389-84db-b56fda86816f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:19.100241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0486c4-368a-4389-84db-b56fda86816f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:19.100530 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0486c4-368a-4389-84db-b56fda86816f None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:19.101234 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0486c4-368a-4389-84db-b56fda86816f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:19.101542 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0486c4-368a-4389-84db-b56fda86816f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:19.103753 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0486c4-368a-4389-84db-b56fda86816f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:19.120962 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 287/570] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:19.120962 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0486c4-368a-4389-84db-b56fda86816f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:19.132982 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae63151-e59f-4b82-b9d4-8aac7a28b47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:19.133793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae63151-e59f-4b82-b9d4-8aac7a28b47e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:19.134080 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae63151-e59f-4b82-b9d4-8aac7a28b47e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:19.134388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae63151-e59f-4b82-b9d4-8aac7a28b47e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:19.145351 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae63151-e59f-4b82-b9d4-8aac7a28b47e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:19.147426 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae63151-e59f-4b82-b9d4-8aac7a28b47e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:19.164623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-3ae63151-e59f-4b82-b9d4-8aac7a28b47e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:19.164623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:19.164623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:19.164623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:19.164623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:19.164623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:19.164623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:19.164623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:19.164623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:19.164623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:19.164623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:19.164623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:19.164623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:19.164623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:19.164623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:19.165660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:19.165660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:19.165660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:19.165660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:19.165660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:19.165660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:19.165660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:19.165660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:19.165660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:19.165660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:19.165660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:19.165660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:19.165660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:19.165660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:19.165660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:19.165660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:19.165660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:19.166545 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:19.166545 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:19.166545 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:19.166545 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:19.170351 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 284/571] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:19.171427 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae63151-e59f-4b82-b9d4-8aac7a28b47e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:19.239470 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0486c4-368a-4389-84db-b56fda86816f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:19.242517 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:19.270399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:19.274735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4917a6-4679-424e-b1fc-4973663daaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:19.275638 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4917a6-4679-424e-b1fc-4973663daaef None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:19.276089 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4917a6-4679-424e-b1fc-4973663daaef None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:19.276651 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4917a6-4679-424e-b1fc-4973663daaef None admin] PATH_INFO: `/v3/roles/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:19.286866 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4917a6-4679-424e-b1fc-4973663daaef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:19.292720 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-ad4917a6-4679-424e-b1fc-4973663daaef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:19.292720 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:19.292720 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:19.292720 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:19.292720 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:19.292720 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:19.292720 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:19.292720 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:19.292720 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:19.292720 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:19.292720 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:19.292720 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:19.292720 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:19.292720 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:19.292720 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:19.292720 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:19.293650 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:19.293650 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:19.293650 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:19.293650 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:19.293650 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:19.293650 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:19.293650 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:19.293650 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:19.293650 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:19.293650 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:19.293650 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:19.293650 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:19.293650 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:19.293650 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:19.293650 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:19.293650 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:19.299747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:19.299747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:19.299747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:19.299747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:19.299747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:19.299747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:19.299747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:19.299747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:19.299747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:19.299747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:19.299747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:19.299747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:19.299747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 288/572] 10.222.0.97 () {56 vars in 1102 bytes} [Tue Jul 26 14:30:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:19.299747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4917a6-4679-424e-b1fc-4973663daaef None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:19.307960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae63151-e59f-4b82-b9d4-8aac7a28b47e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:19.342085 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f08157-eca3-493f-9d0b-24d55892ae12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:19.342693 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f08157-eca3-493f-9d0b-24d55892ae12 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:19.342976 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f08157-eca3-493f-9d0b-24d55892ae12 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:19.343260 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f08157-eca3-493f-9d0b-24d55892ae12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:19.344087 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f08157-eca3-493f-9d0b-24d55892ae12 None admin] RBAC: Adding query filter params (role.id=9074ca9226f8483c885f32fa868a1acc, scope.project.id=c4c8da3011294783b2180ade1bfd9366, user.id=85122c4052fd4ba19a2dae185a707caf) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:19.344400 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f08157-eca3-493f-9d0b-24d55892ae12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:19.345839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f08157-eca3-493f-9d0b-24d55892ae12 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:19.351426 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 285/573] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:30:19 2022] GET /identity/v3/role_assignments?user.id=85122c4052fd4ba19a2dae185a707caf&scope.project.id=c4c8da3011294783b2180ade1bfd9366&role.id=9074ca9226f8483c885f32fa868a1acc => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:19.352204 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f08157-eca3-493f-9d0b-24d55892ae12 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:19.412166 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4917a6-4679-424e-b1fc-4973663daaef None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:19.452950 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd10b2-5c93-4f92-bc21-319b3deb3ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:19.453758 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd10b2-5c93-4f92-bc21-319b3deb3ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:19.454044 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd10b2-5c93-4f92-bc21-319b3deb3ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:19.454317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd10b2-5c93-4f92-bc21-319b3deb3ae2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:19.462879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd10b2-5c93-4f92-bc21-319b3deb3ae2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:19.464382 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd10b2-5c93-4f92-bc21-319b3deb3ae2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:19.464829 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f08157-eca3-493f-9d0b-24d55892ae12 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:19.476759 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-9acd10b2-5c93-4f92-bc21-319b3deb3ae2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 26 14:30:19.476759 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:19.476759 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:19.476759 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:19.476759 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:19.476759 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:19.476759 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:19.476759 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:19.476759 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:19.476759 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:19.476759 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:19.476759 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:19.476759 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:30:19.476759 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:30:19.476759 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:30:19.476759 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:30:19.478445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:19.478445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:19.478445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:30:19.478445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:19.478445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:30:19.478445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:19.478445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:19.478445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:19.478445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:19.478445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:19.478445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:19.478445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:19.478445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:19.478445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:19.478445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:19.478445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:19.478445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:19.478445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:19.479588 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:19.479588 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:19.479588 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:30:19.479588 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:30:19.479588 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:30:19.479588 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:30:19.479588 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:30:19.479588 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:30:19.479588 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 26 14:30:19.479588 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:19.481526 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 289/574] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:30:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:19.482933 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac070a9-cc00-4dca-b576-8b77d2542421 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:19.483218 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac070a9-cc00-4dca-b576-8b77d2542421 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:19.483517 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac070a9-cc00-4dca-b576-8b77d2542421 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:19.487443 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 290/575] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:19.487443 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac070a9-cc00-4dca-b576-8b77d2542421 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:19.506111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:19.513496 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f944f-21de-48e0-81ac-396ae134fa08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:19.513921 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f944f-21de-48e0-81ac-396ae134fa08 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:19.514052 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:19.514217 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f944f-21de-48e0-81ac-396ae134fa08 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:19.514485 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f944f-21de-48e0-81ac-396ae134fa08 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:19.515345 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f944f-21de-48e0-81ac-396ae134fa08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:19.519383 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f944f-21de-48e0-81ac-396ae134fa08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:19.519383 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f944f-21de-48e0-81ac-396ae134fa08 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:19.530767 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 286/576] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:19.539424 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f944f-21de-48e0-81ac-396ae134fa08 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:19.599456 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac070a9-cc00-4dca-b576-8b77d2542421 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:19.635874 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a2a5e-f2d1-4b82-81d7-e16cdd3b2384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:19.636650 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a2a5e-f2d1-4b82-81d7-e16cdd3b2384 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:19.636821 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a2a5e-f2d1-4b82-81d7-e16cdd3b2384 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:19.637094 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a2a5e-f2d1-4b82-81d7-e16cdd3b2384 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:19.637801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307a2a5e-f2d1-4b82-81d7-e16cdd3b2384 None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:19.638104 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307a2a5e-f2d1-4b82-81d7-e16cdd3b2384 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:19.639551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307a2a5e-f2d1-4b82-81d7-e16cdd3b2384 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:19.643988 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f944f-21de-48e0-81ac-396ae134fa08 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:19.654038 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 291/577] 10.222.0.97 () {56 vars in 1103 bytes} [Tue Jul 26 14:30:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:19.654038 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d9c70-e15d-4440-98c9-ee481ead699e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:19.654038 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d9c70-e15d-4440-98c9-ee481ead699e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:19.654038 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d9c70-e15d-4440-98c9-ee481ead699e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:19.671990 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-fc8d9c70-e15d-4440-98c9-ee481ead699e None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:19.673174 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-fc8d9c70-e15d-4440-98c9-ee481ead699e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:19.681692 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4343e3-8a88-4ff1-b25b-b0c3d4f877de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:19.682316 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4343e3-8a88-4ff1-b25b-b0c3d4f877de None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:19.682599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4343e3-8a88-4ff1-b25b-b0c3d4f877de None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:19.682902 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4343e3-8a88-4ff1-b25b-b0c3d4f877de None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:19.683746 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4343e3-8a88-4ff1-b25b-b0c3d4f877de None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:19.684058 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4343e3-8a88-4ff1-b25b-b0c3d4f877de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:19.685468 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4343e3-8a88-4ff1-b25b-b0c3d4f877de None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:19.722780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 287/578] 10.222.0.97 () {56 vars in 1137 bytes} [Tue Jul 26 14:30:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:19.723647 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4343e3-8a88-4ff1-b25b-b0c3d4f877de None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:19.760241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 292/579] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:30:19.761075 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8d9c70-e15d-4440-98c9-ee481ead699e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:19.775997 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:19.782812 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:19.830229 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4343e3-8a88-4ff1-b25b-b0c3d4f877de None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:19.869758 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa9be72-be99-4692-bf1c-83588091adc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:19.870168 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8d9c70-e15d-4440-98c9-ee481ead699e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:19.870407 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9be72-be99-4692-bf1c-83588091adc4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:19.870690 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9be72-be99-4692-bf1c-83588091adc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:19.870963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9be72-be99-4692-bf1c-83588091adc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:19.878463 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa9be72-be99-4692-bf1c-83588091adc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:19.880070 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa9be72-be99-4692-bf1c-83588091adc4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:19.892824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-9aa9be72-be99-4692-bf1c-83588091adc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:19.892824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:19.892824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:19.892824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:19.892824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:19.892824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:19.892824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:19.892824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:19.892824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:19.892824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:19.892824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:19.892824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:19.892824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:30:19.892824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:30:19.892824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:30:19.893694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:30:19.893694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:19.893694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:19.893694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:19.893694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:19.893694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:19.893694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:19.893694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:19.893694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:19.893694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:19.893694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:19.893694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:19.893694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:19.893694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:19.893694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:19.893694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:19.893694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:19.894539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:30:19.894539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:30:19.894539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:30:19.894539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:30:19.894539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:30:19.894539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:30:19.894539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:19.894539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:19.894539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 288/580] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:30:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:19.895038 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa9be72-be99-4692-bf1c-83588091adc4 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:19.910162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ba380b-90ae-4d38-a14c-b8224961ec29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:19.910815 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba380b-90ae-4d38-a14c-b8224961ec29 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:19.911101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba380b-90ae-4d38-a14c-b8224961ec29 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:19.911378 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba380b-90ae-4d38-a14c-b8224961ec29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:19.921898 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ba380b-90ae-4d38-a14c-b8224961ec29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:19.923353 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ba380b-90ae-4d38-a14c-b8224961ec29 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:19.938686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-18ba380b-90ae-4d38-a14c-b8224961ec29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:19.938686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:19.938686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:19.938686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:19.938686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:19.938686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:19.938686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:19.938686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:19.938686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:19.938686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:19.938686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:19.938686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:19.938686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:19.938686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:19.938686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:19.939597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:19.939597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:19.939597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:19.939597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:19.939597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:19.939597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:19.939597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:19.939597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:19.939597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:19.939597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:19.939597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:19.939597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:19.939597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:19.939597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:19.939597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:19.939597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:19.939597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:19.940582 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:19.940582 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:19.940582 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:19.940582 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:19.940582 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 293/581] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:19.940845 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ba380b-90ae-4d38-a14c-b8224961ec29 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:20.016093 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:20.019755 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa9be72-be99-4692-bf1c-83588091adc4 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:20.028864 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:20.058626 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ba380b-90ae-4d38-a14c-b8224961ec29 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:20.059506 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0dff1a-fa1b-481f-8e5d-45f7b8efe4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:20.060222 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0dff1a-fa1b-481f-8e5d-45f7b8efe4cc None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:20.060595 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0dff1a-fa1b-481f-8e5d-45f7b8efe4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:20.060949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0dff1a-fa1b-481f-8e5d-45f7b8efe4cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:20.071584 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0dff1a-fa1b-481f-8e5d-45f7b8efe4cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:20.073109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0dff1a-fa1b-481f-8e5d-45f7b8efe4cc None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:20.087542 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-3c0dff1a-fa1b-481f-8e5d-45f7b8efe4cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:20.087542 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:20.087542 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:20.087542 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:20.087542 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:20.087542 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:20.087542 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:20.087542 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:20.087542 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:20.087542 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:20.087542 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:20.087542 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:20.087542 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:20.087542 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:20.087542 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:20.088618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:20.088618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:20.088618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:20.088618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:20.088618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:20.088618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:20.088618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:20.088618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:20.088618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:20.088618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:20.088618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:20.088618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:20.088618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:20.088618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:20.088618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:20.088618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:20.088618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:20.089707 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:20.089707 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:20.089707 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:20.089707 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:20.096770 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 289/582] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:20.097577 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0dff1a-fa1b-481f-8e5d-45f7b8efe4cc None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:20.099027 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958dd08b-0d00-41dc-a772-4542ba3ce34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:20.099788 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958dd08b-0d00-41dc-a772-4542ba3ce34e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:20.100160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958dd08b-0d00-41dc-a772-4542ba3ce34e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:20.100664 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958dd08b-0d00-41dc-a772-4542ba3ce34e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:20.114139 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958dd08b-0d00-41dc-a772-4542ba3ce34e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:20.123318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-958dd08b-0d00-41dc-a772-4542ba3ce34e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:20.123318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:20.123318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:20.123318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:20.123318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:20.123318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:20.123318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:20.123318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:20.123318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:20.123318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:20.123318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:20.123318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:20.123318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:20.123318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:20.123318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:20.123318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:20.124463 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:20.124463 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:20.124463 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:20.124463 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:20.124463 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:20.124463 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:20.124463 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:20.124463 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:20.124463 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:20.124463 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:20.124463 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:20.124463 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:20.124463 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:20.124463 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:20.124463 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:20.124463 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:20.125659 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:20.125659 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:20.125659 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:20.125659 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:20.125659 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:20.125659 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:20.125659 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:20.125659 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:20.125659 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:20.125659 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:20.125659 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:20.125659 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:20.125659 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 294/583] 10.222.0.97 () {56 vars in 1102 bytes} [Tue Jul 26 14:30:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:20.125659 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958dd08b-0d00-41dc-a772-4542ba3ce34e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:20.210906 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0dff1a-fa1b-481f-8e5d-45f7b8efe4cc None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:20.231909 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958dd08b-0d00-41dc-a772-4542ba3ce34e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:20.246934 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adc2d95-a969-4fdd-9547-4d34c1032f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:20.247713 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc2d95-a969-4fdd-9547-4d34c1032f39 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:20.248100 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc2d95-a969-4fdd-9547-4d34c1032f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:20.248566 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc2d95-a969-4fdd-9547-4d34c1032f39 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:20.249330 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adc2d95-a969-4fdd-9547-4d34c1032f39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:20.249771 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adc2d95-a969-4fdd-9547-4d34c1032f39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:20.251207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adc2d95-a969-4fdd-9547-4d34c1032f39 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:20.260146 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:20.263159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 290/584] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:30:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:20.264139 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adc2d95-a969-4fdd-9547-4d34c1032f39 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:20.273578 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241fa9be-8c98-4725-98a5-7593fb23dbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:20.274144 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241fa9be-8c98-4725-98a5-7593fb23dbbf None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:20.274432 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241fa9be-8c98-4725-98a5-7593fb23dbbf None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:20.274735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241fa9be-8c98-4725-98a5-7593fb23dbbf None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:20.275474 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241fa9be-8c98-4725-98a5-7593fb23dbbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:20.275787 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241fa9be-8c98-4725-98a5-7593fb23dbbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:20.277156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241fa9be-8c98-4725-98a5-7593fb23dbbf None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:20.293986 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 295/585] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:20.293986 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241fa9be-8c98-4725-98a5-7593fb23dbbf None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:20.295379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:20.375290 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adc2d95-a969-4fdd-9547-4d34c1032f39 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:20.403565 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241fa9be-8c98-4725-98a5-7593fb23dbbf None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:20.412045 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b0842-8e5e-4043-af93-2d3de4f75b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:20.412742 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b0842-8e5e-4043-af93-2d3de4f75b4c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:20.413033 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b0842-8e5e-4043-af93-2d3de4f75b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:20.413312 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b0842-8e5e-4043-af93-2d3de4f75b4c None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:20.414157 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b0842-8e5e-4043-af93-2d3de4f75b4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:20.414594 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b0842-8e5e-4043-af93-2d3de4f75b4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:20.416333 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b0842-8e5e-4043-af93-2d3de4f75b4c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:20.427901 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 291/586] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:20.432970 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b0842-8e5e-4043-af93-2d3de4f75b4c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:20.441015 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe6576-6d13-4e15-a636-6fd504e00c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:20.441645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe6576-6d13-4e15-a636-6fd504e00c06 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:20.441928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe6576-6d13-4e15-a636-6fd504e00c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:20.442207 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe6576-6d13-4e15-a636-6fd504e00c06 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:20.442916 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe6576-6d13-4e15-a636-6fd504e00c06 None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:20.443225 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe6576-6d13-4e15-a636-6fd504e00c06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:20.444655 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe6576-6d13-4e15-a636-6fd504e00c06 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:20.453045 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 296/587] 10.222.0.97 () {56 vars in 1103 bytes} [Tue Jul 26 14:30:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:20.453899 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe6576-6d13-4e15-a636-6fd504e00c06 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:20.506801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:20.517732 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:20.537011 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b0842-8e5e-4043-af93-2d3de4f75b4c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:20.559138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe6576-6d13-4e15-a636-6fd504e00c06 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:20.571580 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd9b86-44b7-4c85-8718-b7bea743afb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:20.572491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd9b86-44b7-4c85-8718-b7bea743afb6 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:20.572893 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd9b86-44b7-4c85-8718-b7bea743afb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:20.573262 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd9b86-44b7-4c85-8718-b7bea743afb6 None admin] PATH_INFO: `/v3/projects/c4c8da3011294783b2180ade1bfd9366/users/266abef6cda5495b897170f92043aa03/roles/9074ca9226f8483c885f32fa868a1acc` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:20.588677 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd9b86-44b7-4c85-8718-b7bea743afb6 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4c8da3011294783b2180ade1bfd9366, user_id=266abef6cda5495b897170f92043aa03, role_id=9074ca9226f8483c885f32fa868a1acc)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:20.590180 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd9b86-44b7-4c85-8718-b7bea743afb6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:20.594297 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3fbf0b-ed3e-4e9c-8e9b-42d9945e02d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:20.595000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fbf0b-ed3e-4e9c-8e9b-42d9945e02d1 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:20.595349 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fbf0b-ed3e-4e9c-8e9b-42d9945e02d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:20.595726 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fbf0b-ed3e-4e9c-8e9b-42d9945e02d1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:20.603890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3fbf0b-ed3e-4e9c-8e9b-42d9945e02d1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:20.607265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3fbf0b-ed3e-4e9c-8e9b-42d9945e02d1 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:20.616591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-8c3fbf0b-ed3e-4e9c-8e9b-42d9945e02d1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 14:30:20.616591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:20.616591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:20.616591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:20.616591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:20.616591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:20.616591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:20.616591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:20.616591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:20.616591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:20.616591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:20.616591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:20.616591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:30:20.616591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:30:20.616591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:30:20.616591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:30:20.617646 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:20.617646 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:20.617646 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:30:20.617646 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:20.617646 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:30:20.617646 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:20.617646 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:20.617646 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:20.617646 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:20.617646 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:20.617646 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:20.617646 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:20.617646 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:20.617646 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:20.617646 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:20.617646 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:20.617646 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:20.617646 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:20.618708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:20.618708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:20.618708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:30:20.618708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:30:20.618708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:30:20.618708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:30:20.618708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:30:20.618708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:30:20.618708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 14:30:20.618708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:20.620742 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 297/588] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:30:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:20.620742 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3fbf0b-ed3e-4e9c-8e9b-42d9945e02d1 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:20.659331 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 292/589] 10.222.0.97 () {58 vars in 1439 bytes} [Tue Jul 26 14:30:20 2022] PUT /identity/v3/projects/c4c8da3011294783b2180ade1bfd9366/users/266abef6cda5495b897170f92043aa03/roles/9074ca9226f8483c885f32fa868a1acc => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:30:20.660284 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd9b86-44b7-4c85-8718-b7bea743afb6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:20.804387 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3fbf0b-ed3e-4e9c-8e9b-42d9945e02d1 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:20.804897 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:20.831477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd9b86-44b7-4c85-8718-b7bea743afb6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:20.854464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27abda26-9ca0-4cd5-9914-586f01597db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:20.854464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27abda26-9ca0-4cd5-9914-586f01597db8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:20.854464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27abda26-9ca0-4cd5-9914-586f01597db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:20.854464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27abda26-9ca0-4cd5-9914-586f01597db8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:20.863872 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27abda26-9ca0-4cd5-9914-586f01597db8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:20.866714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27abda26-9ca0-4cd5-9914-586f01597db8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:20.877559 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:20.883749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-27abda26-9ca0-4cd5-9914-586f01597db8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:20.883749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:20.883749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:20.883749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:20.883749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:20.883749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:20.883749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:20.883749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:20.883749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:20.883749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:20.883749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:20.883749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:20.883749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:30:20.883749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:30:20.883749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:30:20.885360 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:30:20.885360 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:20.885360 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:20.885360 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:20.885360 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:20.885360 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:20.885360 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:20.885360 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:20.885360 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:20.885360 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:20.885360 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:20.885360 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:20.885360 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:20.885360 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:20.885360 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:20.885360 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:20.885360 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:20.886429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:30:20.886429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:30:20.886429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:30:20.886429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:30:20.886429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:30:20.886429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:30:20.886429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:20.886429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:20.886429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 298/590] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:30:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:20.886429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27abda26-9ca0-4cd5-9914-586f01597db8 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:20.895425 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7f9f2-fe25-456b-9a39-c255020d15bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:20.895425 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7f9f2-fe25-456b-9a39-c255020d15bc None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:20.895425 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7f9f2-fe25-456b-9a39-c255020d15bc None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:20.895425 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7f9f2-fe25-456b-9a39-c255020d15bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:20.913401 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b7f9f2-fe25-456b-9a39-c255020d15bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:20.919579 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b7f9f2-fe25-456b-9a39-c255020d15bc None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:20.950413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-b3b7f9f2-fe25-456b-9a39-c255020d15bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:20.950413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:20.950413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:20.950413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:20.950413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:20.950413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:20.950413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:20.950413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:20.950413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:20.950413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:20.950413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:20.950413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:20.950413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:20.950413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:20.950413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:20.951561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:20.951561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:20.951561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:20.951561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:20.951561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:20.951561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:20.951561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:20.951561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:20.951561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:20.951561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:20.951561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:20.951561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:20.951561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:20.951561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:20.951561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:20.951561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:20.951561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:20.952626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:20.952626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:20.952626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:20.952626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:20.967021 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 293/591] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:20.967021 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7f9f2-fe25-456b-9a39-c255020d15bc None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:21.079894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27abda26-9ca0-4cd5-9914-586f01597db8 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:21.119180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8583f68-e9f8-40bf-8d8b-7b37a4fe46dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:21.120102 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8583f68-e9f8-40bf-8d8b-7b37a4fe46dc None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:21.121211 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8583f68-e9f8-40bf-8d8b-7b37a4fe46dc None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:21.121799 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8583f68-e9f8-40bf-8d8b-7b37a4fe46dc None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:21.124196 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8583f68-e9f8-40bf-8d8b-7b37a4fe46dc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:21.125450 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8583f68-e9f8-40bf-8d8b-7b37a4fe46dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:21.127028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8583f68-e9f8-40bf-8d8b-7b37a4fe46dc None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:21.159510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7f9f2-fe25-456b-9a39-c255020d15bc None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:21.161827 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 299/592] 10.222.0.97 () {56 vars in 1137 bytes} [Tue Jul 26 14:30:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:21.162930 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8583f68-e9f8-40bf-8d8b-7b37a4fe46dc None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:21.203070 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58945989-3ed5-4d94-a865-2ff740fa16d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:21.204762 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58945989-3ed5-4d94-a865-2ff740fa16d6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:21.205115 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58945989-3ed5-4d94-a865-2ff740fa16d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:21.206232 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58945989-3ed5-4d94-a865-2ff740fa16d6 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:21.207065 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58945989-3ed5-4d94-a865-2ff740fa16d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:21.207631 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58945989-3ed5-4d94-a865-2ff740fa16d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:21.210307 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58945989-3ed5-4d94-a865-2ff740fa16d6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:21.230761 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:21.235296 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 294/593] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:30:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:21.237010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58945989-3ed5-4d94-a865-2ff740fa16d6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:21.304374 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:21.340144 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8583f68-e9f8-40bf-8d8b-7b37a4fe46dc None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:21.371301 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58945989-3ed5-4d94-a865-2ff740fa16d6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:21.389162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e83a4-5f7d-4fda-9339-745d45013bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:21.390719 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e83a4-5f7d-4fda-9339-745d45013bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:21.391538 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e83a4-5f7d-4fda-9339-745d45013bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:21.392036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e83a4-5f7d-4fda-9339-745d45013bb0 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:21.392943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e83a4-5f7d-4fda-9339-745d45013bb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:21.393461 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e83a4-5f7d-4fda-9339-745d45013bb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:21.395885 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e83a4-5f7d-4fda-9339-745d45013bb0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:21.408951 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e918452-2d7f-4945-a5cb-1b9e527017ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:21.409666 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e918452-2d7f-4945-a5cb-1b9e527017ae None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:21.410019 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e918452-2d7f-4945-a5cb-1b9e527017ae None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:21.410363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e918452-2d7f-4945-a5cb-1b9e527017ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:21.413838 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 300/594] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:21.414911 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e83a4-5f7d-4fda-9339-745d45013bb0 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:21.426296 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e918452-2d7f-4945-a5cb-1b9e527017ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:21.427499 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e918452-2d7f-4945-a5cb-1b9e527017ae None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:21.445196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-4e918452-2d7f-4945-a5cb-1b9e527017ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:21.445196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:21.445196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:21.445196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:21.445196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:21.445196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:21.445196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:21.445196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:21.445196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:21.445196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:21.445196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:21.445196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:21.445196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:21.445196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:21.445196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:21.446095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:21.446095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:21.446095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:21.446095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:21.446095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:21.446095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:21.446095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:21.446095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:21.446095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:21.446095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:21.446095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:21.446095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:21.446095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:21.446095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:21.446095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:21.446095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:21.446095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:21.448002 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:21.448002 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:21.448002 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:21.448002 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:21.448002 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 295/595] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:21.449368 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e918452-2d7f-4945-a5cb-1b9e527017ae None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:21.520569 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:21.533933 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e83a4-5f7d-4fda-9339-745d45013bb0 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:21.549481 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:21.557125 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e918452-2d7f-4945-a5cb-1b9e527017ae None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:21.568419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c85109-cdd2-4788-884b-c2b0b9891573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:21.569336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c85109-cdd2-4788-884b-c2b0b9891573 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:21.570244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c85109-cdd2-4788-884b-c2b0b9891573 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:21.570707 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c85109-cdd2-4788-884b-c2b0b9891573 None admin] PATH_INFO: `/v3/projects/c4c8da3011294783b2180ade1bfd9366/users/62adb1f5e9654927847e1717630ff6b9/roles/9074ca9226f8483c885f32fa868a1acc` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:21.585579 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c85109-cdd2-4788-884b-c2b0b9891573 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4c8da3011294783b2180ade1bfd9366, user_id=62adb1f5e9654927847e1717630ff6b9, role_id=9074ca9226f8483c885f32fa868a1acc)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:21.587125 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c85109-cdd2-4788-884b-c2b0b9891573 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:21.590786 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49601399-3157-406b-a338-190ceb072d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:21.591493 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49601399-3157-406b-a338-190ceb072d8a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:21.591796 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49601399-3157-406b-a338-190ceb072d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:21.592054 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49601399-3157-406b-a338-190ceb072d8a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:21.599950 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49601399-3157-406b-a338-190ceb072d8a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:21.601584 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49601399-3157-406b-a338-190ceb072d8a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:21.613235 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-49601399-3157-406b-a338-190ceb072d8a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 14:30:21.613235 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:21.613235 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:21.613235 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:21.613235 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:21.613235 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:21.613235 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:21.613235 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:21.613235 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:21.613235 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:21.613235 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:21.613235 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:21.613235 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:30:21.613235 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:30:21.613235 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:30:21.613235 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:30:21.614319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:21.614319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:21.614319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:30:21.614319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:21.614319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:30:21.614319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:21.614319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:21.614319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:21.614319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:21.614319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:21.614319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:21.614319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:21.614319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:21.614319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:21.614319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:21.614319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:21.614319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:21.614319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:21.615447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:21.615447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:21.615447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:30:21.615447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:30:21.615447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:30:21.615447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:30:21.615447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:30:21.615447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:30:21.615447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 14:30:21.615447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:21.615447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 296/596] 10.222.0.97 () {56 vars in 1102 bytes} [Tue Jul 26 14:30:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:21.620571 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c155061-7ce3-4c5d-bdf2-439866914af7 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:21.620858 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c155061-7ce3-4c5d-bdf2-439866914af7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:21.621134 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c155061-7ce3-4c5d-bdf2-439866914af7 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:21.622399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 297/597] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:21.623109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c155061-7ce3-4c5d-bdf2-439866914af7 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:21.664045 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 301/598] 10.222.0.97 () {58 vars in 1439 bytes} [Tue Jul 26 14:30:21 2022] PUT /identity/v3/projects/c4c8da3011294783b2180ade1bfd9366/users/62adb1f5e9654927847e1717630ff6b9/roles/9074ca9226f8483c885f32fa868a1acc => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:30:21.665736 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c85109-cdd2-4788-884b-c2b0b9891573 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:21.757715 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c155061-7ce3-4c5d-bdf2-439866914af7 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:21.799950 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e546073-80c8-4ae7-b00c-3badcd488ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:21.800593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e546073-80c8-4ae7-b00c-3badcd488ece None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:21.800883 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e546073-80c8-4ae7-b00c-3badcd488ece None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:21.801149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e546073-80c8-4ae7-b00c-3badcd488ece None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:21.802424 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e546073-80c8-4ae7-b00c-3badcd488ece None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:21.802424 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e546073-80c8-4ae7-b00c-3badcd488ece None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:21.803589 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e546073-80c8-4ae7-b00c-3badcd488ece None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:21.817840 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 298/599] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:21.819235 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638c4f0-a11d-4102-ac55-c23a7231e3f9 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:21.819582 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638c4f0-a11d-4102-ac55-c23a7231e3f9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:21.819865 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638c4f0-a11d-4102-ac55-c23a7231e3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:21.821238 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c85109-cdd2-4788-884b-c2b0b9891573 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:21.842523 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-e638c4f0-a11d-4102-ac55-c23a7231e3f9 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:21.843715 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e638c4f0-a11d-4102-ac55-c23a7231e3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:21.857512 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:21.874973 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6aa381-5846-4e02-aa9b-72b605ea6cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:21.875865 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6aa381-5846-4e02-aa9b-72b605ea6cdd None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:21.876147 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6aa381-5846-4e02-aa9b-72b605ea6cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:21.876424 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6aa381-5846-4e02-aa9b-72b605ea6cdd None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:21.877228 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6aa381-5846-4e02-aa9b-72b605ea6cdd None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:21.877539 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6aa381-5846-4e02-aa9b-72b605ea6cdd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:21.880485 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6aa381-5846-4e02-aa9b-72b605ea6cdd None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:21.892792 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:21.909403 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 302/600] 10.222.0.97 () {56 vars in 1139 bytes} [Tue Jul 26 14:30:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:21.911031 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6aa381-5846-4e02-aa9b-72b605ea6cdd None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:21.933751 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 299/601] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:30:21.939427 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e638c4f0-a11d-4102-ac55-c23a7231e3f9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:22.040266 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6aa381-5846-4e02-aa9b-72b605ea6cdd None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:22.072344 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e638c4f0-a11d-4102-ac55-c23a7231e3f9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:22.080313 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b948b2d8-c816-43bb-b63a-93c1e0c1fe78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:22.080970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948b2d8-c816-43bb-b63a-93c1e0c1fe78 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:22.081256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948b2d8-c816-43bb-b63a-93c1e0c1fe78 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:22.081537 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948b2d8-c816-43bb-b63a-93c1e0c1fe78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:22.089443 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b948b2d8-c816-43bb-b63a-93c1e0c1fe78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:22.091096 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b948b2d8-c816-43bb-b63a-93c1e0c1fe78 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:22.104016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-b948b2d8-c816-43bb-b63a-93c1e0c1fe78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:22.104016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:22.104016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:22.104016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:22.104016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:22.104016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:22.104016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:22.104016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:22.104016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:22.104016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:22.104016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:22.104016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:22.104016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:30:22.104016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:30:22.104016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:30:22.104880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:30:22.104880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:22.104880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:22.104880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:22.104880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:22.104880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:22.104880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:22.104880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:22.104880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:22.104880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:22.104880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:22.104880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:22.104880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:22.104880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:22.104880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:22.104880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:22.104880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:22.105743 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:30:22.105743 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:30:22.105743 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:30:22.105743 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:30:22.105743 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:30:22.105743 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:30:22.105743 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:22.105743 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:22.105743 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 303/602] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:30:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:22.106740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b948b2d8-c816-43bb-b63a-93c1e0c1fe78 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:22.119015 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b71ef-bcfe-4fb7-82ff-710fc143677c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:22.119784 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b71ef-bcfe-4fb7-82ff-710fc143677c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:22.120061 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b71ef-bcfe-4fb7-82ff-710fc143677c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:22.120338 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b71ef-bcfe-4fb7-82ff-710fc143677c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:22.131926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b71ef-bcfe-4fb7-82ff-710fc143677c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:22.133477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b71ef-bcfe-4fb7-82ff-710fc143677c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:22.136540 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:22.152694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-df0b71ef-bcfe-4fb7-82ff-710fc143677c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:22.152694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:22.152694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:22.152694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:22.152694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:22.152694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:22.152694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:22.152694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:22.152694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:22.152694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:22.152694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:22.152694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:22.152694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:22.152694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:22.152694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:22.153627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:22.153627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:22.153627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:22.153627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:22.153627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:22.153627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:22.153627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:22.153627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:22.153627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:22.153627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:22.153627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:22.153627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:22.153627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:22.153627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:22.153627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:22.153627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:22.153627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:22.155065 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:22.155065 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:22.155065 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:22.155065 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:22.155065 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 300/603] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:22.155346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b71ef-bcfe-4fb7-82ff-710fc143677c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:22.182446 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:22.239166 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b948b2d8-c816-43bb-b63a-93c1e0c1fe78 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:22.268401 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b71ef-bcfe-4fb7-82ff-710fc143677c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:22.282671 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e67abe-0ed8-4efb-95c5-1f49ea524c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:22.283601 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e67abe-0ed8-4efb-95c5-1f49ea524c3a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:22.283958 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e67abe-0ed8-4efb-95c5-1f49ea524c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:22.284308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e67abe-0ed8-4efb-95c5-1f49ea524c3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:22.298438 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e67abe-0ed8-4efb-95c5-1f49ea524c3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:22.305190 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-c2e67abe-0ed8-4efb-95c5-1f49ea524c3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:22.305190 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:22.305190 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:22.305190 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:22.305190 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:22.305190 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:22.305190 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:22.305190 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:22.305190 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:22.305190 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:22.305190 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:22.305190 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:22.305190 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:22.305190 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:22.305190 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:22.305190 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:22.306181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:22.306181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:22.306181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:22.306181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:22.306181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:22.306181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:22.306181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:22.306181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:22.306181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:22.306181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:22.306181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:22.306181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:22.306181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:22.306181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:22.306181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:22.306181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:22.307021 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:22.307021 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:22.307021 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:22.307021 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:22.307021 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:22.307021 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:22.307021 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:22.307021 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:22.307021 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:22.307021 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:22.307021 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:22.307021 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:22.311675 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 304/604] 10.222.0.97 () {56 vars in 1102 bytes} [Tue Jul 26 14:30:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:22.312334 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26909151-895b-48a6-a931-84481bc9043b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:22.312661 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e67abe-0ed8-4efb-95c5-1f49ea524c3a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:22.313362 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26909151-895b-48a6-a931-84481bc9043b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:22.313822 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26909151-895b-48a6-a931-84481bc9043b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:22.314274 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26909151-895b-48a6-a931-84481bc9043b None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:22.315216 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26909151-895b-48a6-a931-84481bc9043b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:22.315739 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26909151-895b-48a6-a931-84481bc9043b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:22.317631 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26909151-895b-48a6-a931-84481bc9043b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:22.332310 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 301/605] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:30:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:22.333599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26909151-895b-48a6-a931-84481bc9043b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:22.399678 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:22.441104 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e67abe-0ed8-4efb-95c5-1f49ea524c3a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:22.448529 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26909151-895b-48a6-a931-84481bc9043b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:22.460284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:22.480442 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667510c-61d9-4b9c-a75b-15fe638609aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:22.481509 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667510c-61d9-4b9c-a75b-15fe638609aa None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:22.482026 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667510c-61d9-4b9c-a75b-15fe638609aa None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:22.482594 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667510c-61d9-4b9c-a75b-15fe638609aa None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:22.483666 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667510c-61d9-4b9c-a75b-15fe638609aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:22.484197 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667510c-61d9-4b9c-a75b-15fe638609aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:22.485860 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667510c-61d9-4b9c-a75b-15fe638609aa None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:22.492140 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10985f71-3911-47d6-9685-51e24bd67e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:22.492766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10985f71-3911-47d6-9685-51e24bd67e40 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:22.493087 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10985f71-3911-47d6-9685-51e24bd67e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:22.494393 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10985f71-3911-47d6-9685-51e24bd67e40 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:22.495121 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10985f71-3911-47d6-9685-51e24bd67e40 None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:22.495450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10985f71-3911-47d6-9685-51e24bd67e40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:22.497293 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10985f71-3911-47d6-9685-51e24bd67e40 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:22.500512 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 305/606] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:22.501595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667510c-61d9-4b9c-a75b-15fe638609aa None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:22.511749 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 302/607] 10.222.0.97 () {56 vars in 1103 bytes} [Tue Jul 26 14:30:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:22.512621 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10985f71-3911-47d6-9685-51e24bd67e40 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:22.619487 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667510c-61d9-4b9c-a75b-15fe638609aa None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:22.641035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10985f71-3911-47d6-9685-51e24bd67e40 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:22.657419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72369d19-bdd5-4767-b4f6-fbdc83bfc111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:22.658477 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72369d19-bdd5-4767-b4f6-fbdc83bfc111 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:22.658965 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72369d19-bdd5-4767-b4f6-fbdc83bfc111 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:22.659549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72369d19-bdd5-4767-b4f6-fbdc83bfc111 None admin] PATH_INFO: `/v3/projects/c4c8da3011294783b2180ade1bfd9366/users/bad45b307edc415fa6185a82af64ede4/roles/9074ca9226f8483c885f32fa868a1acc` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:22.663017 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:22.678083 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72369d19-bdd5-4767-b4f6-fbdc83bfc111 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4c8da3011294783b2180ade1bfd9366, user_id=bad45b307edc415fa6185a82af64ede4, role_id=9074ca9226f8483c885f32fa868a1acc)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:22.679828 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72369d19-bdd5-4767-b4f6-fbdc83bfc111 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:22.683107 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d42e2-1588-4ac2-ab1d-8d489446d673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:22.684140 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d42e2-1588-4ac2-ab1d-8d489446d673 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:22.684666 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d42e2-1588-4ac2-ab1d-8d489446d673 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:22.685144 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d42e2-1588-4ac2-ab1d-8d489446d673 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:22.693432 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d42e2-1588-4ac2-ab1d-8d489446d673 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:22.696072 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d42e2-1588-4ac2-ab1d-8d489446d673 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:22.707958 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-182d42e2-1588-4ac2-ab1d-8d489446d673 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:22.707958 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:22.707958 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:22.707958 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:22.707958 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:22.707958 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:22.707958 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:22.707958 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:22.707958 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:22.707958 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:22.707958 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:22.707958 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:22.707958 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:30:22.707958 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:30:22.707958 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:30:22.708889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:30:22.708889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:22.708889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:22.708889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:22.708889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:22.708889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:22.708889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:22.708889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:22.708889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:22.708889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:22.708889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:22.708889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:22.708889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:22.708889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:22.708889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:22.708889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:22.708889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:22.709784 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:30:22.709784 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:30:22.709784 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:30:22.709784 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:30:22.709784 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:30:22.709784 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:30:22.709784 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:22.709784 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:22.709784 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 303/608] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:30:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:22.711531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d42e2-1588-4ac2-ab1d-8d489446d673 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:22.793743 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 306/609] 10.222.0.97 () {58 vars in 1439 bytes} [Tue Jul 26 14:30:22 2022] PUT /identity/v3/projects/c4c8da3011294783b2180ade1bfd9366/users/bad45b307edc415fa6185a82af64ede4/roles/9074ca9226f8483c885f32fa868a1acc => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:30:22.794565 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72369d19-bdd5-4767-b4f6-fbdc83bfc111 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:22.824143 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d42e2-1588-4ac2-ab1d-8d489446d673 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:22.854399 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:22.867007 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b2626-f3a0-4e13-9437-f555b7fce903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:22.868175 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b2626-f3a0-4e13-9437-f555b7fce903 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:22.869253 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b2626-f3a0-4e13-9437-f555b7fce903 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:22.869730 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b2626-f3a0-4e13-9437-f555b7fce903 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:22.882512 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2b2626-f3a0-4e13-9437-f555b7fce903 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:22.885021 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2b2626-f3a0-4e13-9437-f555b7fce903 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:22.901899 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-fb2b2626-f3a0-4e13-9437-f555b7fce903 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:22.901899 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:22.901899 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:22.901899 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:22.901899 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:22.901899 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:22.901899 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:22.901899 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:22.901899 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:22.901899 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:22.901899 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:22.901899 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:22.901899 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:22.901899 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:22.901899 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:22.902831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:22.902831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:22.902831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:22.902831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:22.902831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:22.902831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:22.902831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:22.902831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:22.902831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:22.902831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:22.902831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:22.902831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:22.902831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:22.902831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:22.902831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:22.902831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:22.902831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:22.903749 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:22.903749 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:22.903749 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:22.903749 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:22.906050 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 304/610] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:22.907981 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843849e3-2b3e-47da-aba4-963c36c3b518 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:22.908496 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843849e3-2b3e-47da-aba4-963c36c3b518 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:22.909030 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843849e3-2b3e-47da-aba4-963c36c3b518 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:22.911508 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 305/611] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:22.912585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843849e3-2b3e-47da-aba4-963c36c3b518 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:22.946009 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72369d19-bdd5-4767-b4f6-fbdc83bfc111 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:22.948509 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:22.992371 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0060b6d-94f5-40b1-b0d9-af06460e7b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:22.993441 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0060b6d-94f5-40b1-b0d9-af06460e7b62 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:22.993801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0060b6d-94f5-40b1-b0d9-af06460e7b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:22.994083 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0060b6d-94f5-40b1-b0d9-af06460e7b62 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:22.994843 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0060b6d-94f5-40b1-b0d9-af06460e7b62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:22.995154 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0060b6d-94f5-40b1-b0d9-af06460e7b62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:22.996836 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0060b6d-94f5-40b1-b0d9-af06460e7b62 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:23.010744 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 307/612] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:30:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:23.012113 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102c58b-ee2e-4d9f-b4aa-54dad150de8c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:23.012409 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102c58b-ee2e-4d9f-b4aa-54dad150de8c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:23.013304 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102c58b-ee2e-4d9f-b4aa-54dad150de8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:23.034267 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-7102c58b-ee2e-4d9f-b4aa-54dad150de8c None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:23.036475 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7102c58b-ee2e-4d9f-b4aa-54dad150de8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:23.043257 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843849e3-2b3e-47da-aba4-963c36c3b518 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:23.080520 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498a69e-31de-4d6f-97ef-1a226d457070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:23.081136 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498a69e-31de-4d6f-97ef-1a226d457070 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:23.081420 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498a69e-31de-4d6f-97ef-1a226d457070 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:23.081696 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498a69e-31de-4d6f-97ef-1a226d457070 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:23.082389 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8498a69e-31de-4d6f-97ef-1a226d457070 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:23.082699 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8498a69e-31de-4d6f-97ef-1a226d457070 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:23.084100 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8498a69e-31de-4d6f-97ef-1a226d457070 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:23.097042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 306/613] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:23.097909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498a69e-31de-4d6f-97ef-1a226d457070 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:23.139525 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 308/614] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:30:23.139525 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7102c58b-ee2e-4d9f-b4aa-54dad150de8c None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:23.166427 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:23.221987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:23.233063 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498a69e-31de-4d6f-97ef-1a226d457070 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:23.272461 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85480fe1-aca3-420c-89df-5c4c3ae48cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:23.273253 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85480fe1-aca3-420c-89df-5c4c3ae48cec None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:23.273589 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85480fe1-aca3-420c-89df-5c4c3ae48cec None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:23.273941 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85480fe1-aca3-420c-89df-5c4c3ae48cec None admin] PATH_INFO: `/v3/projects/c4c8da3011294783b2180ade1bfd9366/users/c81180fb4ff04c1186a21e43fb597a91/roles/9074ca9226f8483c885f32fa868a1acc` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:23.284894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7102c58b-ee2e-4d9f-b4aa-54dad150de8c None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:23.290601 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85480fe1-aca3-420c-89df-5c4c3ae48cec None admin] RBAC: Authorizing `identity:create_grant(project_id=c4c8da3011294783b2180ade1bfd9366, user_id=c81180fb4ff04c1186a21e43fb597a91, role_id=9074ca9226f8483c885f32fa868a1acc)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:23.292078 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85480fe1-aca3-420c-89df-5c4c3ae48cec None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:23.324223 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b8e98d-f91f-4b99-8200-9ada681a4e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:23.325248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8e98d-f91f-4b99-8200-9ada681a4e51 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:23.325734 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8e98d-f91f-4b99-8200-9ada681a4e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:23.326225 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8e98d-f91f-4b99-8200-9ada681a4e51 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:23.334095 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b8e98d-f91f-4b99-8200-9ada681a4e51 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:23.335814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b8e98d-f91f-4b99-8200-9ada681a4e51 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:23.348317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-26b8e98d-f91f-4b99-8200-9ada681a4e51 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 14:30:23.348317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:23.348317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:23.348317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:23.348317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:23.348317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:23.348317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:23.348317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:23.348317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:23.348317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:23.348317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:23.348317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:23.348317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:30:23.348317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:30:23.348317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:30:23.348317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:30:23.349256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:23.349256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:23.349256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:30:23.349256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:23.349256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:30:23.349256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:23.349256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:23.349256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:23.349256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:23.349256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:23.349256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:23.349256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:23.349256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:23.349256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:23.349256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:23.349256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:23.349256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:23.349256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:23.350161 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:23.350161 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:23.350161 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:30:23.350161 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:30:23.350161 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:30:23.350161 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:30:23.350161 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:30:23.350161 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:30:23.350161 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 14:30:23.350161 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:23.352702 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 309/615] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:30:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:23.353681 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b8e98d-f91f-4b99-8200-9ada681a4e51 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:23.386532 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 307/616] 10.222.0.97 () {58 vars in 1439 bytes} [Tue Jul 26 14:30:23 2022] PUT /identity/v3/projects/c4c8da3011294783b2180ade1bfd9366/users/c81180fb4ff04c1186a21e43fb597a91/roles/9074ca9226f8483c885f32fa868a1acc => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:30:23.387749 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85480fe1-aca3-420c-89df-5c4c3ae48cec None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:23.446415 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:23.473599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b8e98d-f91f-4b99-8200-9ada681a4e51 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:23.508865 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85480fe1-aca3-420c-89df-5c4c3ae48cec None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:23.513261 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473ebaa-5c13-4ad1-a4f5-6bbc385e5762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:23.514232 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473ebaa-5c13-4ad1-a4f5-6bbc385e5762 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:23.514700 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473ebaa-5c13-4ad1-a4f5-6bbc385e5762 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:23.515174 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473ebaa-5c13-4ad1-a4f5-6bbc385e5762 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:23.528483 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7473ebaa-5c13-4ad1-a4f5-6bbc385e5762 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:23.534680 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-7473ebaa-5c13-4ad1-a4f5-6bbc385e5762 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:23.534680 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:23.534680 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:23.534680 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:23.534680 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:23.534680 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:23.534680 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:23.534680 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:23.534680 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:23.534680 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:23.534680 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:23.534680 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:23.534680 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:23.534680 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:23.534680 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:23.534680 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:23.535708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:23.535708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:23.535708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:23.535708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:23.535708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:23.535708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:23.535708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:23.535708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:23.535708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:23.535708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:23.535708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:23.535708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:23.535708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:23.535708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:23.535708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:23.535708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:23.536520 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:23.536520 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:23.536520 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:23.536520 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:23.536520 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:23.536520 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:23.536520 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:23.536520 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:23.536520 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:23.536520 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:23.536520 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:23.536520 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:23.539173 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 310/617] 10.222.0.97 () {56 vars in 1102 bytes} [Tue Jul 26 14:30:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:23.545336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473ebaa-5c13-4ad1-a4f5-6bbc385e5762 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:23.548143 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03378371-dd77-4ea7-ae49-7770fdc2853e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:23.548863 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03378371-dd77-4ea7-ae49-7770fdc2853e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:23.549035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03378371-dd77-4ea7-ae49-7770fdc2853e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:23.549312 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03378371-dd77-4ea7-ae49-7770fdc2853e None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:23.550123 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03378371-dd77-4ea7-ae49-7770fdc2853e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:23.551110 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03378371-dd77-4ea7-ae49-7770fdc2853e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:23.551892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03378371-dd77-4ea7-ae49-7770fdc2853e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:23.561980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:23.573329 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 308/618] 10.222.0.97 () {56 vars in 1135 bytes} [Tue Jul 26 14:30:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:23.581173 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03378371-dd77-4ea7-ae49-7770fdc2853e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:23.659730 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473ebaa-5c13-4ad1-a4f5-6bbc385e5762 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:23.697571 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86877ad-5dd5-4aad-8636-7adad5809e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:23.698464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86877ad-5dd5-4aad-8636-7adad5809e6e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:23.698893 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86877ad-5dd5-4aad-8636-7adad5809e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:23.699328 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86877ad-5dd5-4aad-8636-7adad5809e6e None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:23.700245 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86877ad-5dd5-4aad-8636-7adad5809e6e None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:23.700698 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86877ad-5dd5-4aad-8636-7adad5809e6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:23.702212 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86877ad-5dd5-4aad-8636-7adad5809e6e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:23.706141 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:23.713230 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03378371-dd77-4ea7-ae49-7770fdc2853e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:23.713725 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 311/619] 10.222.0.97 () {56 vars in 1103 bytes} [Tue Jul 26 14:30:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:30:23.719263 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86877ad-5dd5-4aad-8636-7adad5809e6e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:23.751355 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5fd00-7940-47cf-b31d-0057fd8dc40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:23.751962 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5fd00-7940-47cf-b31d-0057fd8dc40b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:23.752246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5fd00-7940-47cf-b31d-0057fd8dc40b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:23.752531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5fd00-7940-47cf-b31d-0057fd8dc40b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:23.764050 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda5fd00-7940-47cf-b31d-0057fd8dc40b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:23.765482 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda5fd00-7940-47cf-b31d-0057fd8dc40b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:23.781834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-cda5fd00-7940-47cf-b31d-0057fd8dc40b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:23.781834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:23.781834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:23.781834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:23.781834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:23.781834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:23.781834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:23.781834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:23.781834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:23.781834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:23.781834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:23.781834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:23.781834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:23.781834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:23.781834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:23.782992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:23.782992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:23.782992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:23.782992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:23.782992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:23.782992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:23.782992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:23.782992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:23.782992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:23.782992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:23.782992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:23.782992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:23.782992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:23.782992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:23.782992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:23.782992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:23.782992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:23.784085 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:23.784085 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:23.784085 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:23.784085 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:23.787954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 309/620] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:30:23.793220 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5fd00-7940-47cf-b31d-0057fd8dc40b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:23.835032 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86877ad-5dd5-4aad-8636-7adad5809e6e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:23.860657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:23.872972 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1309d9ad-a966-40bd-afee-99ea7ccf60f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:23.878799 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309d9ad-a966-40bd-afee-99ea7ccf60f4 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:23.878799 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309d9ad-a966-40bd-afee-99ea7ccf60f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:23.878799 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309d9ad-a966-40bd-afee-99ea7ccf60f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:23.886056 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1309d9ad-a966-40bd-afee-99ea7ccf60f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:23.887541 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1309d9ad-a966-40bd-afee-99ea7ccf60f4 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:23.902791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-1309d9ad-a966-40bd-afee-99ea7ccf60f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:23.902791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:23.902791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:23.902791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:23.902791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:23.902791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:23.902791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:23.902791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:23.902791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:23.902791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:23.902791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:23.902791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:23.902791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:23.902791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:23.902791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:23.903684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:23.903684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:23.903684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:23.903684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:23.903684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:23.903684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:23.903684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:23.903684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:23.903684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:23.903684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:23.903684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:23.903684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:23.903684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:23.903684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:23.903684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:23.903684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:23.903684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:23.904544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:23.904544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:23.904544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:23.904544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:23.904544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 312/621] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:23.905529 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c471279-a70b-40bc-bd70-1c59a1708f60 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:23.909695 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c471279-a70b-40bc-bd70-1c59a1708f60 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:23.909695 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c471279-a70b-40bc-bd70-1c59a1708f60 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:23.911340 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 313/622] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:23.912123 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c471279-a70b-40bc-bd70-1c59a1708f60 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:23.913702 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5fd00-7940-47cf-b31d-0057fd8dc40b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:23.951119 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df16d77-4c7c-495d-bf18-5137287f3223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:23.951692 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df16d77-4c7c-495d-bf18-5137287f3223 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:23.951978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df16d77-4c7c-495d-bf18-5137287f3223 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:23.952255 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df16d77-4c7c-495d-bf18-5137287f3223 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:23.952949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df16d77-4c7c-495d-bf18-5137287f3223 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:23.953255 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df16d77-4c7c-495d-bf18-5137287f3223 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:23.954663 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df16d77-4c7c-495d-bf18-5137287f3223 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:23.966708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:23.967557 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 310/623] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:30:23.968883 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f158ae-093f-4731-bf2a-b230cb9424ed None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:23.969226 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f158ae-093f-4731-bf2a-b230cb9424ed None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:23.969509 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f158ae-093f-4731-bf2a-b230cb9424ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:23.988375 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-10f158ae-093f-4731-bf2a-b230cb9424ed None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:23.989597 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-10f158ae-093f-4731-bf2a-b230cb9424ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:24.033586 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c471279-a70b-40bc-bd70-1c59a1708f60 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:24.071845 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befab747-eadb-4c94-b50a-bf440782d1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:24.072490 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befab747-eadb-4c94-b50a-bf440782d1fa None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:24.072777 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befab747-eadb-4c94-b50a-bf440782d1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:24.073133 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befab747-eadb-4c94-b50a-bf440782d1fa None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:24.073834 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befab747-eadb-4c94-b50a-bf440782d1fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:24.074139 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befab747-eadb-4c94-b50a-bf440782d1fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:24.075604 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befab747-eadb-4c94-b50a-bf440782d1fa None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:24.077874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 311/624] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:30:24.078956 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f158ae-093f-4731-bf2a-b230cb9424ed None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:24.098348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 314/625] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:24.104882 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befab747-eadb-4c94-b50a-bf440782d1fa None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:24.125665 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:24.189767 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f158ae-093f-4731-bf2a-b230cb9424ed None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:24.225682 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704f401-2b6f-49a1-878a-d60c26a70177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:24.226700 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704f401-2b6f-49a1-878a-d60c26a70177 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:24.227203 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704f401-2b6f-49a1-878a-d60c26a70177 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:24.227785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704f401-2b6f-49a1-878a-d60c26a70177 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:24.228945 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befab747-eadb-4c94-b50a-bf440782d1fa None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:24.235044 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c704f401-2b6f-49a1-878a-d60c26a70177 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:24.236787 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c704f401-2b6f-49a1-878a-d60c26a70177 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:24.241127 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:24.250380 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-c704f401-2b6f-49a1-878a-d60c26a70177 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:24.250380 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:24.250380 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:24.250380 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:24.250380 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:24.250380 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:24.250380 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:24.250380 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:24.250380 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:24.250380 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:24.250380 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:24.250380 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:24.250380 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:30:24.250380 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:30:24.250380 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:30:24.251266 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:30:24.251266 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:24.251266 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:24.251266 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:24.251266 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:24.251266 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:24.251266 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:24.251266 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:24.251266 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:24.251266 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:24.251266 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:24.251266 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:24.251266 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:24.251266 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:24.251266 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:24.251266 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:24.251266 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:24.252281 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:30:24.252281 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:30:24.252281 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:30:24.252281 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:30:24.252281 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:30:24.252281 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:30:24.252281 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:24.252281 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:24.254346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 312/626] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:30:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:24.255297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704f401-2b6f-49a1-878a-d60c26a70177 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:24.265962 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e5b8c3-398c-4e21-95f8-52997f1d3a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:24.267054 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5b8c3-398c-4e21-95f8-52997f1d3a79 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:24.267208 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5b8c3-398c-4e21-95f8-52997f1d3a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:24.267427 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5b8c3-398c-4e21-95f8-52997f1d3a79 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:24.276516 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e5b8c3-398c-4e21-95f8-52997f1d3a79 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:24.277974 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e5b8c3-398c-4e21-95f8-52997f1d3a79 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:24.289464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-45e5b8c3-398c-4e21-95f8-52997f1d3a79 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:30:24.289464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:24.289464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:24.289464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:24.289464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:24.289464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:24.289464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:24.289464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:24.289464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:24.289464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:24.289464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:24.289464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:24.289464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:30:24.289464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:30:24.289464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:30:24.289464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:30:24.291232 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:24.291232 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:24.291232 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:30:24.291232 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:24.291232 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:30:24.291232 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:24.291232 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:24.291232 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:24.291232 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:24.291232 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:24.291232 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:24.291232 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:24.291232 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:24.291232 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:24.291232 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:24.291232 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:24.291232 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:24.291232 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:24.292227 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:24.292227 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:24.292227 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:30:24.292227 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:30:24.292227 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:30:24.292227 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:30:24.292227 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:30:24.292227 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:30:24.292227 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:30:24.292227 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:24.295835 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 315/627] 10.222.0.97 () {56 vars in 1093 bytes} [Tue Jul 26 14:30:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:30:24.297034 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e5b8c3-398c-4e21-95f8-52997f1d3a79 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:24.362860 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704f401-2b6f-49a1-878a-d60c26a70177 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:24.375630 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:24.399860 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d3537f-a674-41e9-b69e-f555832e11a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:24.404292 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3537f-a674-41e9-b69e-f555832e11a7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:24.404292 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3537f-a674-41e9-b69e-f555832e11a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:24.404292 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3537f-a674-41e9-b69e-f555832e11a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:24.404542 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e5b8c3-398c-4e21-95f8-52997f1d3a79 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:24.412249 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d3537f-a674-41e9-b69e-f555832e11a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:24.417541 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-04d3537f-a674-41e9-b69e-f555832e11a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:24.417541 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:24.417541 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:24.417541 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:24.417541 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:24.417541 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:24.417541 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:24.417541 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:24.417541 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:24.417541 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:24.417541 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:24.417541 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:24.417541 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:24.417541 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:24.417541 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:24.417541 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:24.418552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:24.418552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:24.418552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:24.418552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:24.418552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:24.418552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:24.418552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:24.418552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:24.418552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:24.418552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:24.418552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:24.418552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:24.418552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:24.418552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:24.418552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:24.418552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:24.419568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:24.419568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:24.419568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:24.419568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:24.419568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:24.419568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:24.419568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:24.419568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:24.419568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:24.419568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:24.419568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:24.419568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:24.421846 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 313/628] 10.222.0.97 () {56 vars in 1102 bytes} [Tue Jul 26 14:30:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:24.423282 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d3537f-a674-41e9-b69e-f555832e11a7 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:24.445866 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14acbdd-af6f-4b99-9d00-16fcd4dfc170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:24.446758 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14acbdd-af6f-4b99-9d00-16fcd4dfc170 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:24.447429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14acbdd-af6f-4b99-9d00-16fcd4dfc170 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:24.447429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14acbdd-af6f-4b99-9d00-16fcd4dfc170 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:24.448338 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14acbdd-af6f-4b99-9d00-16fcd4dfc170 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:24.448776 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14acbdd-af6f-4b99-9d00-16fcd4dfc170 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:24.450394 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14acbdd-af6f-4b99-9d00-16fcd4dfc170 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:24.463294 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 316/629] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:30:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:24.464470 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14acbdd-af6f-4b99-9d00-16fcd4dfc170 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:24.500087 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:24.532605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d3537f-a674-41e9-b69e-f555832e11a7 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:24.567326 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ceb6f-1ea3-4e6b-a7bc-9209088abf5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:24.567980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ceb6f-1ea3-4e6b-a7bc-9209088abf5c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:24.568264 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ceb6f-1ea3-4e6b-a7bc-9209088abf5c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:24.568538 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ceb6f-1ea3-4e6b-a7bc-9209088abf5c None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:24.569334 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146ceb6f-1ea3-4e6b-a7bc-9209088abf5c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:24.569641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146ceb6f-1ea3-4e6b-a7bc-9209088abf5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:24.571065 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146ceb6f-1ea3-4e6b-a7bc-9209088abf5c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:24.588314 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14acbdd-af6f-4b99-9d00-16fcd4dfc170 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:24.593521 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 314/630] 10.222.0.97 () {56 vars in 1133 bytes} [Tue Jul 26 14:30:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:24.593521 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ceb6f-1ea3-4e6b-a7bc-9209088abf5c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:24.619843 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:24.634067 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f34b5-0461-4268-8d89-7d96301953da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:24.635068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f34b5-0461-4268-8d89-7d96301953da None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:24.635609 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f34b5-0461-4268-8d89-7d96301953da None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:24.636117 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f34b5-0461-4268-8d89-7d96301953da None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:24.637064 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f34b5-0461-4268-8d89-7d96301953da None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:24.637591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f34b5-0461-4268-8d89-7d96301953da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:24.639340 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f34b5-0461-4268-8d89-7d96301953da None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:24.653914 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 317/631] 10.222.0.97 () {56 vars in 1103 bytes} [Tue Jul 26 14:30:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:24.655165 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f34b5-0461-4268-8d89-7d96301953da None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:24.696160 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ceb6f-1ea3-4e6b-a7bc-9209088abf5c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:24.731079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731919ab-8a86-4306-98cf-dc88afba6957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:24.731788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731919ab-8a86-4306-98cf-dc88afba6957 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:24.732073 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731919ab-8a86-4306-98cf-dc88afba6957 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:24.732351 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731919ab-8a86-4306-98cf-dc88afba6957 None admin] PATH_INFO: `/v3/projects/c4c8da3011294783b2180ade1bfd9366/users/85122c4052fd4ba19a2dae185a707caf/roles/9074ca9226f8483c885f32fa868a1acc` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:24.747982 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731919ab-8a86-4306-98cf-dc88afba6957 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4c8da3011294783b2180ade1bfd9366, user_id=85122c4052fd4ba19a2dae185a707caf, role_id=9074ca9226f8483c885f32fa868a1acc)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:24.749625 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731919ab-8a86-4306-98cf-dc88afba6957 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:24.757474 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:24.768309 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f34b5-0461-4268-8d89-7d96301953da None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:24.804794 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167f04f-e355-49fd-8248-250e147af9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:24.805428 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167f04f-e355-49fd-8248-250e147af9a1 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:24.806323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167f04f-e355-49fd-8248-250e147af9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:24.806609 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167f04f-e355-49fd-8248-250e147af9a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:24.858508 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 315/632] 10.222.0.97 () {58 vars in 1439 bytes} [Tue Jul 26 14:30:24 2022] PUT /identity/v3/projects/c4c8da3011294783b2180ade1bfd9366/users/85122c4052fd4ba19a2dae185a707caf/roles/9074ca9226f8483c885f32fa868a1acc => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:30:24.859287 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731919ab-8a86-4306-98cf-dc88afba6957 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:24.966491 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d167f04f-e355-49fd-8248-250e147af9a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:24.968009 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d167f04f-e355-49fd-8248-250e147af9a1 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:25.055480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:25.101089 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731919ab-8a86-4306-98cf-dc88afba6957 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:25.154769 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19da481-4e77-41b7-8962-e075f08db7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:25.155684 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19da481-4e77-41b7-8962-e075f08db7a1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:25.156136 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19da481-4e77-41b7-8962-e075f08db7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:25.156526 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19da481-4e77-41b7-8962-e075f08db7a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:25.171320 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19da481-4e77-41b7-8962-e075f08db7a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:25.173848 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19da481-4e77-41b7-8962-e075f08db7a1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:25.200832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-e19da481-4e77-41b7-8962-e075f08db7a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:25.200832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:25.200832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:25.200832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:25.200832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:25.200832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:25.200832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:25.200832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:25.200832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:25.200832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:25.200832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:25.200832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:25.200832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:25.200832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:25.200832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:25.201766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:25.201766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:25.201766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:25.201766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:25.201766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:25.201766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:25.201766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:25.201766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:25.201766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:25.201766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:25.201766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:25.201766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:25.201766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:25.201766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:25.201766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:25.201766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:25.201766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:25.202647 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:25.202647 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:25.202647 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:25.202647 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:25.202647 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 316/633] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:25.212336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19da481-4e77-41b7-8962-e075f08db7a1 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:25.238573 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:25.254828 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 318/634] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:30:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:30:25.260915 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167f04f-e355-49fd-8248-250e147af9a1 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:25.372503 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19da481-4e77-41b7-8962-e075f08db7a1 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:25.414961 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:25.448122 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15033b75-b53b-4fa8-8c53-48134bf4d5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:25.448753 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15033b75-b53b-4fa8-8c53-48134bf4d5dd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:25.449032 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15033b75-b53b-4fa8-8c53-48134bf4d5dd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:25.449303 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15033b75-b53b-4fa8-8c53-48134bf4d5dd None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:25.450023 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15033b75-b53b-4fa8-8c53-48134bf4d5dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:25.450332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15033b75-b53b-4fa8-8c53-48134bf4d5dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:25.451924 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15033b75-b53b-4fa8-8c53-48134bf4d5dd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:25.485905 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167f04f-e355-49fd-8248-250e147af9a1 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:25.503788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 317/635] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:25.503788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c70917-6944-4d45-b562-371d13fc8587 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:25.503788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c70917-6944-4d45-b562-371d13fc8587 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:25.503788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c70917-6944-4d45-b562-371d13fc8587 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:25.503788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 318/636] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:25.510846 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187a338-bfb8-43b7-978a-c7f3758ea3e0 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:25.511126 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187a338-bfb8-43b7-978a-c7f3758ea3e0 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:25.511430 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187a338-bfb8-43b7-978a-c7f3758ea3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:25.546541 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a187a338-bfb8-43b7-978a-c7f3758ea3e0 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:25.547884 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a187a338-bfb8-43b7-978a-c7f3758ea3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:25.568039 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6b285-6f15-4081-b24f-4482519470d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:25.568701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6b285-6f15-4081-b24f-4482519470d9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:25.568981 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6b285-6f15-4081-b24f-4482519470d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:25.569253 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6b285-6f15-4081-b24f-4482519470d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:25.590082 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a6b285-6f15-4081-b24f-4482519470d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:25.594743 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a6b285-6f15-4081-b24f-4482519470d9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:25.637514 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-68a6b285-6f15-4081-b24f-4482519470d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:25.637514 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:25.637514 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:25.637514 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:25.637514 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:25.637514 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:25.637514 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:25.637514 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:25.637514 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:25.637514 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:25.637514 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:25.637514 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:25.637514 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:25.637514 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:25.637514 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:25.639151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:25.639151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:25.639151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:25.639151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:25.639151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:25.639151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:25.639151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:25.639151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:25.639151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:25.639151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:25.639151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:25.639151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:25.639151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:25.639151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:25.639151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:25.639151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:25.639151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:25.640053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:25.640053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:25.640053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:25.640053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:25.655789 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 319/637] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:30:25.656924 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6b285-6f15-4081-b24f-4482519470d9 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:25.664054 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 319/638] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:30:25.666086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187a338-bfb8-43b7-978a-c7f3758ea3e0 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:25.788810 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:25.819741 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6b285-6f15-4081-b24f-4482519470d9 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:25.823826 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187a338-bfb8-43b7-978a-c7f3758ea3e0 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:25.837334 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:25.862396 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5290ea2c-df96-4402-bfa8-bd3d37a8fc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:25.863161 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290ea2c-df96-4402-bfa8-bd3d37a8fc18 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:25.863557 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290ea2c-df96-4402-bfa8-bd3d37a8fc18 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:25.863915 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290ea2c-df96-4402-bfa8-bd3d37a8fc18 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:25.867092 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf68f96-4048-4712-b088-a86cb14a3262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:25.867768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf68f96-4048-4712-b088-a86cb14a3262 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:25.868554 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf68f96-4048-4712-b088-a86cb14a3262 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:25.868957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf68f96-4048-4712-b088-a86cb14a3262 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:25.869664 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf68f96-4048-4712-b088-a86cb14a3262 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:25.869968 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf68f96-4048-4712-b088-a86cb14a3262 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:25.871359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf68f96-4048-4712-b088-a86cb14a3262 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:25.873415 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5290ea2c-df96-4402-bfa8-bd3d37a8fc18 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:25.874908 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5290ea2c-df96-4402-bfa8-bd3d37a8fc18 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:25.885830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 320/639] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:25.887094 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf68f96-4048-4712-b088-a86cb14a3262 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:25.890061 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-5290ea2c-df96-4402-bfa8-bd3d37a8fc18 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 26 14:30:25.890061 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:25.890061 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:25.890061 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:25.890061 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:25.890061 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:25.890061 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:25.890061 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:25.890061 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:25.890061 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:25.890061 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:25.890061 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:25.890061 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:30:25.890061 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:30:25.890061 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:30:25.890061 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:30:25.891207 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:25.891207 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:25.891207 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:30:25.891207 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:25.891207 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:30:25.891207 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:25.891207 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:25.891207 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:25.891207 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:25.891207 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:25.891207 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:25.891207 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:25.891207 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:25.891207 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:25.891207 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:25.891207 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:25.891207 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:25.891207 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:25.892912 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:25.892912 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:25.892912 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:30:25.892912 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:30:25.892912 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:30:25.892912 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:30:25.892912 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:30:25.892912 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:30:25.892912 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 26 14:30:25.892912 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:25.896015 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 320/640] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:30:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:25.897249 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74edb8ed-186d-42f2-b1b9-caa7f63467b3 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:25.897537 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74edb8ed-186d-42f2-b1b9-caa7f63467b3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:25.897817 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74edb8ed-186d-42f2-b1b9-caa7f63467b3 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:25.903897 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 321/641] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:25.903897 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74edb8ed-186d-42f2-b1b9-caa7f63467b3 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:26.008175 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf68f96-4048-4712-b088-a86cb14a3262 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:26.013248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74edb8ed-186d-42f2-b1b9-caa7f63467b3 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:26.016280 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:26.052057 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4343b1be-4785-4a71-95ce-d1856ee34a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:26.052704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4343b1be-4785-4a71-95ce-d1856ee34a15 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:26.052995 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4343b1be-4785-4a71-95ce-d1856ee34a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:26.053244 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4343b1be-4785-4a71-95ce-d1856ee34a15 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:26.060782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e523ab-6890-45b7-93f8-24c26c7b1862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:26.061481 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e523ab-6890-45b7-93f8-24c26c7b1862 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:26.061760 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e523ab-6890-45b7-93f8-24c26c7b1862 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:26.062086 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e523ab-6890-45b7-93f8-24c26c7b1862 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:26.065325 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4343b1be-4785-4a71-95ce-d1856ee34a15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:26.069363 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e523ab-6890-45b7-93f8-24c26c7b1862 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:26.070720 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-4343b1be-4785-4a71-95ce-d1856ee34a15 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:26.070720 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:26.070720 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:26.070720 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:26.070720 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:26.070720 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:26.070720 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:26.070720 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:26.070720 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:26.070720 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:26.070720 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:26.070720 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:26.070720 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:26.070720 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:26.070720 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:26.070720 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:26.072332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:26.072332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:26.072332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:26.072332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:26.072332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:26.072332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:26.072332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:26.072332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:26.072332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:26.072332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:26.072332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:26.072332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:26.072332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:26.072332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:26.072332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:26.072332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:26.078460 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:26.078460 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:26.078460 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:26.078460 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:26.078460 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:26.078460 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:26.078460 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:26.078460 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:26.078460 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:26.078460 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:26.078460 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:26.078460 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:26.079077 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e523ab-6890-45b7-93f8-24c26c7b1862 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:26.079151 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 321/642] 10.222.0.97 () {56 vars in 1102 bytes} [Tue Jul 26 14:30:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:26.079151 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4343b1be-4785-4a71-95ce-d1856ee34a15 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:26.090467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-d0e523ab-6890-45b7-93f8-24c26c7b1862 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:26.090467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:26.090467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:26.090467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:26.090467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:26.090467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:26.090467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:26.090467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:26.090467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:26.090467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:26.090467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:26.090467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:26.090467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:30:26.090467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:30:26.090467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:30:26.091380 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:30:26.091380 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:26.091380 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:26.091380 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:26.091380 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:26.091380 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:26.091380 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:26.091380 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:26.091380 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:26.091380 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:26.091380 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:26.091380 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:26.091380 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:26.091380 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:26.091380 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:26.091380 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:26.091380 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:26.092387 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:30:26.092387 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:30:26.092387 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:30:26.092387 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:30:26.092387 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:30:26.092387 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:30:26.092387 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:26.092387 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:26.092387 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 322/643] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:30:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:26.094058 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2d8bc-e645-4896-b402-10109800b43d None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:26.094410 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2d8bc-e645-4896-b402-10109800b43d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:26.094757 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2d8bc-e645-4896-b402-10109800b43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:26.113483 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-f3e2d8bc-e645-4896-b402-10109800b43d None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:26.114828 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f3e2d8bc-e645-4896-b402-10109800b43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:26.116083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:26.195895 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 323/644] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:30:26.196708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e2d8bc-e645-4896-b402-10109800b43d None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:26.214748 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4343b1be-4785-4a71-95ce-d1856ee34a15 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:26.255477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7241f-7de8-41b2-a9f9-a055fcac30d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:26.256169 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7241f-7de8-41b2-a9f9-a055fcac30d1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:26.256689 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7241f-7de8-41b2-a9f9-a055fcac30d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:26.256689 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7241f-7de8-41b2-a9f9-a055fcac30d1 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:26.257454 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e7241f-7de8-41b2-a9f9-a055fcac30d1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:26.257767 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e7241f-7de8-41b2-a9f9-a055fcac30d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:26.259787 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e7241f-7de8-41b2-a9f9-a055fcac30d1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:26.286675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 322/645] 10.222.0.97 () {56 vars in 1137 bytes} [Tue Jul 26 14:30:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:26.286675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7241f-7de8-41b2-a9f9-a055fcac30d1 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:26.387113 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:26.397313 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e2d8bc-e645-4896-b402-10109800b43d None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:26.425931 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:26.449521 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7241f-7de8-41b2-a9f9-a055fcac30d1 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:26.464334 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d9b64-058e-429f-935d-7650fbd5d9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:26.465209 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d9b64-058e-429f-935d-7650fbd5d9aa None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:26.465561 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d9b64-058e-429f-935d-7650fbd5d9aa None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:26.465866 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d9b64-058e-429f-935d-7650fbd5d9aa None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:26.466583 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d9b64-058e-429f-935d-7650fbd5d9aa None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:26.466894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d9b64-058e-429f-935d-7650fbd5d9aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:26.468337 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d9b64-058e-429f-935d-7650fbd5d9aa None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:26.477688 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 324/646] 10.222.0.97 () {56 vars in 1103 bytes} [Tue Jul 26 14:30:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:26.478546 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d9b64-058e-429f-935d-7650fbd5d9aa None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:26.510098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f75753-c823-47a1-af26-05a9bd10184f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:26.511072 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f75753-c823-47a1-af26-05a9bd10184f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:26.511707 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f75753-c823-47a1-af26-05a9bd10184f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:26.512051 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f75753-c823-47a1-af26-05a9bd10184f None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:26.512892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f75753-c823-47a1-af26-05a9bd10184f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:26.513286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f75753-c823-47a1-af26-05a9bd10184f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:26.514833 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f75753-c823-47a1-af26-05a9bd10184f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:26.529584 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 323/647] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:30:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:26.534492 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f75753-c823-47a1-af26-05a9bd10184f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:26.641456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f75753-c823-47a1-af26-05a9bd10184f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:26.668573 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d9b64-058e-429f-935d-7650fbd5d9aa None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:26.684764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7260c9d-5b45-4f61-ac9e-caf44361c6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:26.685497 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7260c9d-5b45-4f61-ac9e-caf44361c6a3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:26.685913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7260c9d-5b45-4f61-ac9e-caf44361c6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:26.686300 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7260c9d-5b45-4f61-ac9e-caf44361c6a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:26.708451 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:26.708701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d1168-5171-437b-8c31-371acf74c7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:26.709141 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d1168-5171-437b-8c31-371acf74c7cd None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:26.709708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d1168-5171-437b-8c31-371acf74c7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:26.710115 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d1168-5171-437b-8c31-371acf74c7cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:26.722388 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d1168-5171-437b-8c31-371acf74c7cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:26.728747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-ef1d1168-5171-437b-8c31-371acf74c7cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:26.728747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:26.728747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:26.728747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:26.728747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:26.728747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:26.728747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:26.728747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:26.728747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:26.728747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:26.728747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:26.728747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:26.728747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:26.728747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:26.728747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:26.728747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:26.729695 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:26.729695 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:26.729695 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:26.729695 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:26.729695 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:26.729695 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:26.729695 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:26.729695 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:26.729695 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:26.729695 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:26.729695 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:26.729695 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:26.729695 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:26.729695 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:26.729695 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:26.729695 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:26.730554 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:26.730554 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:26.730554 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:26.730554 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:26.730554 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:26.730554 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:26.730554 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:26.730554 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:26.730554 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:26.730554 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:26.730554 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:26.730554 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:26.734938 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 325/648] 10.222.0.97 () {56 vars in 1102 bytes} [Tue Jul 26 14:30:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:26.736080 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d1168-5171-437b-8c31-371acf74c7cd None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:26.754969 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:26.808918 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7260c9d-5b45-4f61-ac9e-caf44361c6a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:26.812508 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7260c9d-5b45-4f61-ac9e-caf44361c6a3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:26.881582 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d1168-5171-437b-8c31-371acf74c7cd None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:26.928300 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef9075-3d84-47f6-b813-e09e232e0362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:26.929323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef9075-3d84-47f6-b813-e09e232e0362 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:26.929323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef9075-3d84-47f6-b813-e09e232e0362 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:26.930121 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef9075-3d84-47f6-b813-e09e232e0362 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:26.945443 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eef9075-3d84-47f6-b813-e09e232e0362 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:26.947573 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eef9075-3d84-47f6-b813-e09e232e0362 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:26.963118 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 324/649] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:30:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:30:26.964145 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7260c9d-5b45-4f61-ac9e-caf44361c6a3 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:26.969333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-1eef9075-3d84-47f6-b813-e09e232e0362 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:26.969333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:26.969333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:26.969333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:26.969333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:26.969333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:26.969333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:26.969333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:26.969333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:26.969333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:26.969333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:26.969333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:26.969333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:26.969333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:26.969333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:26.970375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:26.970375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:26.970375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:26.970375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:26.970375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:26.970375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:26.970375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:26.970375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:26.970375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:26.970375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:26.970375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:26.970375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:26.970375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:26.970375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:26.970375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:26.970375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:26.970375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:26.971584 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:26.971584 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:26.971584 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:26.971584 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:26.973622 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 326/650] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:26.974720 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef9075-3d84-47f6-b813-e09e232e0362 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:27.055531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:27.086310 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:27.133246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef9075-3d84-47f6-b813-e09e232e0362 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:27.137649 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7260c9d-5b45-4f61-ac9e-caf44361c6a3 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:27.174635 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba60c9-0c15-4a9f-8dae-c8dbaf825b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:27.175274 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba60c9-0c15-4a9f-8dae-c8dbaf825b78 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:27.175598 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba60c9-0c15-4a9f-8dae-c8dbaf825b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:27.175879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba60c9-0c15-4a9f-8dae-c8dbaf825b78 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:27.176597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcba60c9-0c15-4a9f-8dae-c8dbaf825b78 None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:27.177134 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcba60c9-0c15-4a9f-8dae-c8dbaf825b78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:27.178800 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcba60c9-0c15-4a9f-8dae-c8dbaf825b78 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:27.193121 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 327/651] 10.222.0.97 () {56 vars in 1103 bytes} [Tue Jul 26 14:30:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:27.193121 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba60c9-0c15-4a9f-8dae-c8dbaf825b78 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:27.203795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc7d0c0-5261-4cc1-b870-ee5a44154a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:27.204780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7d0c0-5261-4cc1-b870-ee5a44154a4f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:27.205305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7d0c0-5261-4cc1-b870-ee5a44154a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:27.205753 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7d0c0-5261-4cc1-b870-ee5a44154a4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:27.207100 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc7d0c0-5261-4cc1-b870-ee5a44154a4f None admin] RBAC: Adding query filter params (role.id=9074ca9226f8483c885f32fa868a1acc, scope.project.id=c4c8da3011294783b2180ade1bfd9366, user.id=266abef6cda5495b897170f92043aa03) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:27.207643 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc7d0c0-5261-4cc1-b870-ee5a44154a4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:27.210103 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc7d0c0-5261-4cc1-b870-ee5a44154a4f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:27.231521 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 325/652] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:30:26 2022] GET /identity/v3/role_assignments?user.id=266abef6cda5495b897170f92043aa03&scope.project.id=c4c8da3011294783b2180ade1bfd9366&role.id=9074ca9226f8483c885f32fa868a1acc => generated 608 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:27.231521 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc7d0c0-5261-4cc1-b870-ee5a44154a4f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:27.335057 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba60c9-0c15-4a9f-8dae-c8dbaf825b78 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:27.374155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:27.380560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6df067-f2f6-495e-aa9a-2d2ad71cfea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:27.381227 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6df067-f2f6-495e-aa9a-2d2ad71cfea2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:27.381507 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6df067-f2f6-495e-aa9a-2d2ad71cfea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:27.381778 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6df067-f2f6-495e-aa9a-2d2ad71cfea2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:27.394317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6df067-f2f6-495e-aa9a-2d2ad71cfea2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:27.395816 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6df067-f2f6-495e-aa9a-2d2ad71cfea2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:27.414124 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-bc6df067-f2f6-495e-aa9a-2d2ad71cfea2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:27.414124 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:27.414124 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:27.414124 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:27.414124 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:27.414124 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:27.414124 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:27.414124 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:27.414124 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:27.414124 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:27.414124 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:27.414124 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:27.414124 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:27.414124 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:27.414124 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:27.415254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:27.415254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:27.415254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:27.415254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:27.415254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:27.415254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:27.415254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:27.415254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:27.415254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:27.415254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:27.415254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:27.415254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:27.415254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:27.415254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:27.415254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:27.415254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:27.415254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:27.416378 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:27.416378 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:27.416378 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:27.416378 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:27.416378 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 328/653] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:27.416710 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6df067-f2f6-495e-aa9a-2d2ad71cfea2 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:27.431734 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc7d0c0-5261-4cc1-b870-ee5a44154a4f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:27.447557 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:27.477937 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5de2c-f528-4652-a092-8beca2370e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:27.478926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5de2c-f528-4652-a092-8beca2370e92 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:27.479428 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5de2c-f528-4652-a092-8beca2370e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:27.479964 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5de2c-f528-4652-a092-8beca2370e92 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:27.481372 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5de2c-f528-4652-a092-8beca2370e92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:27.481827 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5de2c-f528-4652-a092-8beca2370e92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:27.483456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5de2c-f528-4652-a092-8beca2370e92 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:27.500707 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 326/654] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:27.502509 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7cdb5b-7354-4ed8-8120-fd2008ee18e5 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:27.502975 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7cdb5b-7354-4ed8-8120-fd2008ee18e5 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:27.503243 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7cdb5b-7354-4ed8-8120-fd2008ee18e5 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:27.504706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 327/655] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:27.505876 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7cdb5b-7354-4ed8-8120-fd2008ee18e5 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:27.555011 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6df067-f2f6-495e-aa9a-2d2ad71cfea2 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:27.605474 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f796c17c-e5f0-4bec-bd6d-0d0fe72aa24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:27.606391 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796c17c-e5f0-4bec-bd6d-0d0fe72aa24e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:27.606843 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796c17c-e5f0-4bec-bd6d-0d0fe72aa24e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:27.607749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796c17c-e5f0-4bec-bd6d-0d0fe72aa24e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:27.620098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7cdb5b-7354-4ed8-8120-fd2008ee18e5 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:27.623246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f796c17c-e5f0-4bec-bd6d-0d0fe72aa24e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:27.624920 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f796c17c-e5f0-4bec-bd6d-0d0fe72aa24e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:27.644523 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-f796c17c-e5f0-4bec-bd6d-0d0fe72aa24e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:27.644523 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:27.644523 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:27.644523 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:27.644523 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:27.644523 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:27.644523 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:27.644523 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:27.644523 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:27.644523 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:27.644523 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:27.644523 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:27.644523 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:27.644523 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:27.644523 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:27.645639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:27.645639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:27.645639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:27.645639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:27.645639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:27.645639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:27.645639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:27.645639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:27.645639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:27.645639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:27.645639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:27.645639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:27.645639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:27.645639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:27.645639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:27.645639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:27.645639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:27.646711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:27.646711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:27.646711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:27.646711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:27.646711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 329/656] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:27.651572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f796c17c-e5f0-4bec-bd6d-0d0fe72aa24e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:27.660623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9883cdf-a154-4a1e-bbc5-276307694038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:27.661410 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9883cdf-a154-4a1e-bbc5-276307694038 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:27.661797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9883cdf-a154-4a1e-bbc5-276307694038 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:27.662451 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9883cdf-a154-4a1e-bbc5-276307694038 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:27.662948 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9883cdf-a154-4a1e-bbc5-276307694038 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:27.663325 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9883cdf-a154-4a1e-bbc5-276307694038 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:27.665505 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9883cdf-a154-4a1e-bbc5-276307694038 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:27.684797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 328/657] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:27.684797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c33eb-aedd-4848-96a5-b18778e81c70 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:27.684797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c33eb-aedd-4848-96a5-b18778e81c70 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:27.684797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c33eb-aedd-4848-96a5-b18778e81c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:27.703717 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:27.710355 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-797c33eb-aedd-4848-96a5-b18778e81c70 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:27.713163 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-797c33eb-aedd-4848-96a5-b18778e81c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:27.746789 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:27.784287 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f796c17c-e5f0-4bec-bd6d-0d0fe72aa24e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:27.808005 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 329/658] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:30:27.808657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c33eb-aedd-4848-96a5-b18778e81c70 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:27.830084 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35497bd-2edc-454c-ab4d-fecef52777c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:27.831028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35497bd-2edc-454c-ab4d-fecef52777c2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:27.831544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35497bd-2edc-454c-ab4d-fecef52777c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:27.832061 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35497bd-2edc-454c-ab4d-fecef52777c2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:27.840715 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35497bd-2edc-454c-ab4d-fecef52777c2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:27.844245 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35497bd-2edc-454c-ab4d-fecef52777c2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:27.859025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-f35497bd-2edc-454c-ab4d-fecef52777c2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 14:30:27.859025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:27.859025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:27.859025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:27.859025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:27.859025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:27.859025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:27.859025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:27.859025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:27.859025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:27.859025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:27.859025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:27.859025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:30:27.859025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:30:27.859025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:30:27.859025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:30:27.860226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:27.860226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:27.860226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:30:27.860226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:27.860226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:30:27.860226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:27.860226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:27.860226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:27.860226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:27.860226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:27.860226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:27.860226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:27.860226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:27.860226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:27.860226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:27.860226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:27.860226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:27.860226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:27.861178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:27.861178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:27.861178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:30:27.861178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:30:27.861178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:30:27.861178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:30:27.861178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:30:27.861178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:30:27.861178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 14:30:27.861178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:27.861178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 330/659] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:30:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:27.861754 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35497bd-2edc-454c-ab4d-fecef52777c2 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:27.941026 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c33eb-aedd-4848-96a5-b18778e81c70 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:27.982570 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82445ef-036c-4f4a-8b6c-67bb9d57a2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:27.983210 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82445ef-036c-4f4a-8b6c-67bb9d57a2ca None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:27.983525 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82445ef-036c-4f4a-8b6c-67bb9d57a2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:27.983805 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82445ef-036c-4f4a-8b6c-67bb9d57a2ca None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:27.984522 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82445ef-036c-4f4a-8b6c-67bb9d57a2ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:27.984826 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82445ef-036c-4f4a-8b6c-67bb9d57a2ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:27.985221 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:27.986205 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82445ef-036c-4f4a-8b6c-67bb9d57a2ca None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:27.994351 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35497bd-2edc-454c-ab4d-fecef52777c2 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:28.004601 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 330/660] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:28.005569 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82445ef-036c-4f4a-8b6c-67bb9d57a2ca None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:28.033930 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:28.040477 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac3b80-aa2f-4b1f-a4ee-5e2575193139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:28.041305 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac3b80-aa2f-4b1f-a4ee-5e2575193139 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:28.041693 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac3b80-aa2f-4b1f-a4ee-5e2575193139 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:28.042073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac3b80-aa2f-4b1f-a4ee-5e2575193139 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:28.053905 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac3b80-aa2f-4b1f-a4ee-5e2575193139 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:28.055037 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac3b80-aa2f-4b1f-a4ee-5e2575193139 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:28.068391 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-d3ac3b80-aa2f-4b1f-a4ee-5e2575193139 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:28.068391 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:28.068391 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:28.068391 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:28.068391 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:28.068391 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:28.068391 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:28.068391 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:28.068391 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:28.068391 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:28.068391 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:28.068391 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:28.068391 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:30:28.068391 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:30:28.068391 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:30:28.069336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:30:28.069336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:28.069336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:28.069336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:28.069336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:28.069336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:28.069336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:28.069336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:28.069336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:28.069336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:28.069336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:28.069336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:28.069336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:28.069336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:28.069336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:28.069336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:28.069336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:28.070952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:30:28.070952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:30:28.070952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:30:28.070952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:30:28.070952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:30:28.070952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:30:28.070952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:28.070952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:28.070952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 331/661] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:30:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:28.070952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac3b80-aa2f-4b1f-a4ee-5e2575193139 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:28.139304 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82445ef-036c-4f4a-8b6c-67bb9d57a2ca None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:28.184704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ae17b-6ed0-4453-a981-daa08a70fe6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:28.185801 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ae17b-6ed0-4453-a981-daa08a70fe6b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:28.186276 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ae17b-6ed0-4453-a981-daa08a70fe6b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:28.186776 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ae17b-6ed0-4453-a981-daa08a70fe6b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:28.200615 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9ae17b-6ed0-4453-a981-daa08a70fe6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:28.207149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-da9ae17b-6ed0-4453-a981-daa08a70fe6b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:28.207149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:28.207149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:28.207149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:28.207149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:28.207149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:28.207149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:28.207149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:28.207149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:28.207149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:28.207149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:28.207149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:28.207149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:28.207149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:28.207149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:28.207149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:28.208469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:28.208469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:28.208469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:28.208469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:28.208469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:28.208469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:28.208469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:28.208469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:28.208469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:28.208469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:28.208469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:28.208469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:28.208469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:28.208469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:28.208469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:28.208469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:28.209517 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:28.209517 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:28.209517 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:28.209517 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:28.209517 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:28.209517 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:28.209517 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:28.209517 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:28.209517 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:28.209517 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:28.209517 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:28.209517 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:28.210342 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac3b80-aa2f-4b1f-a4ee-5e2575193139 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:28.212437 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 331/662] 10.222.0.97 () {56 vars in 1102 bytes} [Tue Jul 26 14:30:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:28.213401 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ae17b-6ed0-4453-a981-daa08a70fe6b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:28.261168 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c879206-b483-46bb-96d6-93c5f6bf2905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:28.261841 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c879206-b483-46bb-96d6-93c5f6bf2905 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:28.262269 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c879206-b483-46bb-96d6-93c5f6bf2905 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:28.262634 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c879206-b483-46bb-96d6-93c5f6bf2905 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:28.263887 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c879206-b483-46bb-96d6-93c5f6bf2905 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:28.264293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c879206-b483-46bb-96d6-93c5f6bf2905 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:28.266646 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c879206-b483-46bb-96d6-93c5f6bf2905 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:28.290947 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 332/663] 10.222.0.97 () {56 vars in 1137 bytes} [Tue Jul 26 14:30:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:28.292018 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c879206-b483-46bb-96d6-93c5f6bf2905 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:28.310367 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:28.333304 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:28.353664 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ae17b-6ed0-4453-a981-daa08a70fe6b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:28.391956 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce520de3-e092-421e-8480-bb6a64fd3050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:28.392593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce520de3-e092-421e-8480-bb6a64fd3050 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:28.392876 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce520de3-e092-421e-8480-bb6a64fd3050 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:28.393150 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce520de3-e092-421e-8480-bb6a64fd3050 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:28.402766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce520de3-e092-421e-8480-bb6a64fd3050 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:28.404260 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce520de3-e092-421e-8480-bb6a64fd3050 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:28.416110 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-ce520de3-e092-421e-8480-bb6a64fd3050 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 14:30:28.416110 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:28.416110 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:28.416110 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:28.416110 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:28.416110 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:28.416110 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:28.416110 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:28.416110 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:28.416110 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:28.416110 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:28.416110 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:28.416110 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:30:28.416110 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:30:28.416110 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:30:28.416110 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:30:28.417237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:28.417237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:28.417237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:30:28.417237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:28.417237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:30:28.417237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:28.417237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:28.417237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:28.417237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:28.417237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:28.417237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:28.417237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:28.417237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:28.417237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:28.417237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:28.417237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:28.417237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:28.417237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:28.418239 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:28.418239 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:28.418239 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:30:28.418239 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:30:28.418239 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:30:28.418239 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:30:28.418239 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:30:28.418239 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:30:28.418239 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 14:30:28.418239 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:28.421844 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 332/664] 10.222.0.97 () {56 vars in 1102 bytes} [Tue Jul 26 14:30:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:28.422682 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce520de3-e092-421e-8480-bb6a64fd3050 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:28.444048 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c879206-b483-46bb-96d6-93c5f6bf2905 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:28.502972 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201fa25c-3be5-43cd-99fc-b45125c174d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:28.504052 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201fa25c-3be5-43cd-99fc-b45125c174d8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:28.504524 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201fa25c-3be5-43cd-99fc-b45125c174d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:28.504988 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201fa25c-3be5-43cd-99fc-b45125c174d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:28.506320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201fa25c-3be5-43cd-99fc-b45125c174d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:28.506842 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201fa25c-3be5-43cd-99fc-b45125c174d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:28.510015 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201fa25c-3be5-43cd-99fc-b45125c174d8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:28.532233 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 333/665] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:30:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:28.539375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201fa25c-3be5-43cd-99fc-b45125c174d8 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:28.546465 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce520de3-e092-421e-8480-bb6a64fd3050 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:28.585049 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be42051-bb3b-4eca-8f2e-1675979b27e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:28.585711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be42051-bb3b-4eca-8f2e-1675979b27e1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:28.586006 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be42051-bb3b-4eca-8f2e-1675979b27e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:28.586296 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be42051-bb3b-4eca-8f2e-1675979b27e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:28.587624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be42051-bb3b-4eca-8f2e-1675979b27e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:28.587936 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be42051-bb3b-4eca-8f2e-1675979b27e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:28.589335 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be42051-bb3b-4eca-8f2e-1675979b27e1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:28.598028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 333/666] 10.222.0.97 () {56 vars in 1103 bytes} [Tue Jul 26 14:30:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:28.598876 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be42051-bb3b-4eca-8f2e-1675979b27e1 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:28.605860 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:28.628864 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:28.666298 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201fa25c-3be5-43cd-99fc-b45125c174d8 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:28.708588 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4aea3-fe90-4b50-9bfe-01a182e064f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:28.709220 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4aea3-fe90-4b50-9bfe-01a182e064f0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:28.709502 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4aea3-fe90-4b50-9bfe-01a182e064f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:28.709818 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4aea3-fe90-4b50-9bfe-01a182e064f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:28.724070 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be42051-bb3b-4eca-8f2e-1675979b27e1 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:28.770786 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e67457-4121-400a-a711-052a05521eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:28.771881 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e67457-4121-400a-a711-052a05521eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:28.772578 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e67457-4121-400a-a711-052a05521eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:28.773278 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e67457-4121-400a-a711-052a05521eb9 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:28.774129 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e67457-4121-400a-a711-052a05521eb9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:28.774439 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e67457-4121-400a-a711-052a05521eb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:28.775909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e67457-4121-400a-a711-052a05521eb9 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:28.789468 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba4aea3-fe90-4b50-9bfe-01a182e064f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:28.791016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba4aea3-fe90-4b50-9bfe-01a182e064f0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:28.800563 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 334/667] 10.222.0.97 () {56 vars in 1139 bytes} [Tue Jul 26 14:30:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:28.801510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e67457-4121-400a-a711-052a05521eb9 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:28.877298 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:28.881758 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:28.911440 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 334/668] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:30:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:30:28.916850 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4aea3-fe90-4b50-9bfe-01a182e064f0 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:28.925581 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e67457-4121-400a-a711-052a05521eb9 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:28.967850 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e56c41-134b-43e7-b9c0-b74cb72ff45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:28.968887 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e56c41-134b-43e7-b9c0-b74cb72ff45f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:28.969237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e56c41-134b-43e7-b9c0-b74cb72ff45f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:28.969588 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e56c41-134b-43e7-b9c0-b74cb72ff45f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:28.970463 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e56c41-134b-43e7-b9c0-b74cb72ff45f None admin] RBAC: Adding query filter params (role.id=9074ca9226f8483c885f32fa868a1acc, scope.project.id=c4c8da3011294783b2180ade1bfd9366, user.id=62adb1f5e9654927847e1717630ff6b9) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:28.970857 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e56c41-134b-43e7-b9c0-b74cb72ff45f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:28.972637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e56c41-134b-43e7-b9c0-b74cb72ff45f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:28.983638 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 335/669] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:30:28 2022] GET /identity/v3/role_assignments?user.id=62adb1f5e9654927847e1717630ff6b9&scope.project.id=c4c8da3011294783b2180ade1bfd9366&role.id=9074ca9226f8483c885f32fa868a1acc => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:28.985297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e56c41-134b-43e7-b9c0-b74cb72ff45f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:29.067977 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4aea3-fe90-4b50-9bfe-01a182e064f0 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:29.120417 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a7f39-d037-407f-9808-34ebd12f5cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:29.121198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a7f39-d037-407f-9808-34ebd12f5cee None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:29.121607 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a7f39-d037-407f-9808-34ebd12f5cee None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:29.121990 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a7f39-d037-407f-9808-34ebd12f5cee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:29.136583 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08a7f39-d037-407f-9808-34ebd12f5cee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:29.138131 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08a7f39-d037-407f-9808-34ebd12f5cee None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:29.162279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-c08a7f39-d037-407f-9808-34ebd12f5cee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:29.162279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:29.162279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:29.162279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:29.162279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:29.162279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:29.162279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:29.162279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:29.162279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:29.162279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:29.162279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:29.162279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:29.162279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:29.162279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:29.162279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:29.163230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:29.163230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:29.163230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:29.163230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:29.163230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:29.163230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:29.163230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:29.163230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:29.163230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:29.163230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:29.163230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:29.163230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:29.163230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:29.163230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:29.163230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:29.163230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:29.163230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:29.164171 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:29.164171 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:29.164171 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:29.164171 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:29.164171 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 335/670] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:29.170004 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a7f39-d037-407f-9808-34ebd12f5cee None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:29.170194 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e56c41-134b-43e7-b9c0-b74cb72ff45f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:29.212320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:29.214791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:29.227933 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2552f-f516-4731-bc13-e84a2f8ca875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:29.229773 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2552f-f516-4731-bc13-e84a2f8ca875 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:29.230056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2552f-f516-4731-bc13-e84a2f8ca875 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:29.230333 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2552f-f516-4731-bc13-e84a2f8ca875 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:29.326665 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a7f39-d037-407f-9808-34ebd12f5cee None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:29.335204 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb2552f-f516-4731-bc13-e84a2f8ca875 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:29.337592 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb2552f-f516-4731-bc13-e84a2f8ca875 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:29.367502 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d543a9-f3b8-4c95-aec6-14628189da6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:29.369117 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d543a9-f3b8-4c95-aec6-14628189da6e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:29.369613 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d543a9-f3b8-4c95-aec6-14628189da6e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:29.370097 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d543a9-f3b8-4c95-aec6-14628189da6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:29.384535 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d543a9-f3b8-4c95-aec6-14628189da6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:29.386462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d543a9-f3b8-4c95-aec6-14628189da6e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:29.404599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-a8d543a9-f3b8-4c95-aec6-14628189da6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:29.404599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:29.404599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:29.404599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:29.404599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:29.404599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:29.404599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:29.404599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:29.404599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:29.404599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:29.404599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:29.404599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:29.404599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:29.404599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:29.404599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:29.405560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:29.405560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:29.405560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:29.405560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:29.405560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:29.405560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:29.405560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:29.405560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:29.405560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:29.405560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:29.405560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:29.405560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:29.405560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:29.405560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:29.405560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:29.405560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:29.405560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:29.406434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:29.406434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:29.406434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:29.406434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:29.408848 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 336/671] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:29.414476 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d543a9-f3b8-4c95-aec6-14628189da6e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:29.504449 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:29.530575 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d543a9-f3b8-4c95-aec6-14628189da6e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:29.570133 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70259990-b46f-4300-87bf-04127a0835fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:29.571594 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70259990-b46f-4300-87bf-04127a0835fe None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:29.572217 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70259990-b46f-4300-87bf-04127a0835fe None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:29.572340 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70259990-b46f-4300-87bf-04127a0835fe None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:29.573160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70259990-b46f-4300-87bf-04127a0835fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:29.573568 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70259990-b46f-4300-87bf-04127a0835fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:29.575119 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70259990-b46f-4300-87bf-04127a0835fe None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:29.592684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 337/672] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:29.592684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70259990-b46f-4300-87bf-04127a0835fe None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:29.606487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 336/673] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:30:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:30:29.612743 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2552f-f516-4731-bc13-e84a2f8ca875 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:29.655801 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:29.702354 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70259990-b46f-4300-87bf-04127a0835fe None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:29.740037 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2552f-f516-4731-bc13-e84a2f8ca875 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:29.744921 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7ea78-2c77-48e8-b485-884ae63c6885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:29.745558 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7ea78-2c77-48e8-b485-884ae63c6885 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:29.745847 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7ea78-2c77-48e8-b485-884ae63c6885 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:29.746188 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7ea78-2c77-48e8-b485-884ae63c6885 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:29.746823 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e7ea78-2c77-48e8-b485-884ae63c6885 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:29.747139 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e7ea78-2c77-48e8-b485-884ae63c6885 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:29.748557 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e7ea78-2c77-48e8-b485-884ae63c6885 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:29.752257 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:29.767561 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 338/674] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:29.767561 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7ea78-2c77-48e8-b485-884ae63c6885 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:29.789533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624173a4-8801-423b-82b1-0a83d3ec9e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:29.790985 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624173a4-8801-423b-82b1-0a83d3ec9e5a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:29.791151 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624173a4-8801-423b-82b1-0a83d3ec9e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:29.791562 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624173a4-8801-423b-82b1-0a83d3ec9e5a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:29.803615 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624173a4-8801-423b-82b1-0a83d3ec9e5a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:29.806772 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624173a4-8801-423b-82b1-0a83d3ec9e5a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:29.823425 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-624173a4-8801-423b-82b1-0a83d3ec9e5a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 14:30:29.823425 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:29.823425 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:29.823425 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:29.823425 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:29.823425 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:29.823425 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:29.823425 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:29.823425 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:29.823425 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:29.823425 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:29.823425 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:29.823425 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:30:29.823425 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:30:29.823425 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:30:29.823425 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:30:29.824404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:29.824404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:29.824404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:30:29.824404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:29.824404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:30:29.824404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:29.824404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:29.824404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:29.824404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:29.824404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:29.824404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:29.824404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:29.824404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:29.824404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:29.824404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:29.824404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:29.824404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:29.824404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:29.825349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:29.825349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:29.825349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:30:29.825349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:30:29.825349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:30:29.825349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:30:29.825349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:30:29.825349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:30:29.825349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 14:30:29.825349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:29.825349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 337/675] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:30:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:29.827222 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742e192-bea7-4539-9890-547ce80dc129 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:29.827797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742e192-bea7-4539-9890-547ce80dc129 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:29.828125 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742e192-bea7-4539-9890-547ce80dc129 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:29.829967 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 338/676] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:29.835293 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2742e192-bea7-4539-9890-547ce80dc129 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:29.904871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7ea78-2c77-48e8-b485-884ae63c6885 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:29.948577 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:29.953773 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399fca03-ab3f-4f5b-8e89-5a2f48068485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:29.954486 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399fca03-ab3f-4f5b-8e89-5a2f48068485 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:29.961011 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399fca03-ab3f-4f5b-8e89-5a2f48068485 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:29.961011 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399fca03-ab3f-4f5b-8e89-5a2f48068485 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:29.966662 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2742e192-bea7-4539-9890-547ce80dc129 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:29.975944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399fca03-ab3f-4f5b-8e89-5a2f48068485 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:29.977949 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399fca03-ab3f-4f5b-8e89-5a2f48068485 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:29.999704 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-399fca03-ab3f-4f5b-8e89-5a2f48068485 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:29.999704 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:29.999704 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:29.999704 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:29.999704 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:29.999704 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:29.999704 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:29.999704 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:29.999704 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:29.999704 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:29.999704 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:29.999704 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:29.999704 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:29.999704 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:29.999704 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:30.000662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:30.000662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:30.000662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:30.000662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:30.000662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:30.000662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:30.000662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:30.000662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:30.000662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:30.000662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:30.000662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:30.000662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:30.000662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:30.000662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:30.000662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:30.000662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:30.000662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:30.001535 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:30.001535 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:30.001535 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:30.001535 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:30.006135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 339/677] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:30.007134 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399fca03-ab3f-4f5b-8e89-5a2f48068485 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:30.015032 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc415996-e726-43bd-83ac-84e97eb3c8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:30.015743 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc415996-e726-43bd-83ac-84e97eb3c8f2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:30.016324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc415996-e726-43bd-83ac-84e97eb3c8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:30.016618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc415996-e726-43bd-83ac-84e97eb3c8f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:30.026825 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc415996-e726-43bd-83ac-84e97eb3c8f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:30.028466 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc415996-e726-43bd-83ac-84e97eb3c8f2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:30.054745 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-fc415996-e726-43bd-83ac-84e97eb3c8f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:30.054745 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:30.054745 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:30.054745 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:30.054745 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:30.054745 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:30.054745 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:30.054745 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:30.054745 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:30.054745 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:30.054745 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:30.054745 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:30.054745 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:30:30.054745 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:30:30.054745 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:30:30.055903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:30:30.055903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:30.055903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:30.055903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:30.055903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:30.055903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:30.055903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:30.055903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:30.055903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:30.055903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:30.055903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:30.055903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:30.055903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:30.055903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:30.055903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:30.055903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:30.055903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:30.056803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:30:30.056803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:30:30.056803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:30:30.056803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:30:30.056803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:30:30.056803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:30:30.056803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:30.056803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:30.057284 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 339/678] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:30:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:30.058986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8746114-619f-458b-8175-82f4546d51eb None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:30.059390 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8746114-619f-458b-8175-82f4546d51eb None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:30.059844 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8746114-619f-458b-8175-82f4546d51eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:30.081429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:30.085338 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-b8746114-619f-458b-8175-82f4546d51eb None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:30.087602 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b8746114-619f-458b-8175-82f4546d51eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:30.142340 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399fca03-ab3f-4f5b-8e89-5a2f48068485 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:30.180724 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 340/679] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:30:30.181354 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8746114-619f-458b-8175-82f4546d51eb None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:30.213474 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e4c69-3b4b-48f5-af08-53a818966494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:30.214211 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e4c69-3b4b-48f5-af08-53a818966494 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:30.214425 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e4c69-3b4b-48f5-af08-53a818966494 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:30.214744 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e4c69-3b4b-48f5-af08-53a818966494 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:30.217460 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96e4c69-3b4b-48f5-af08-53a818966494 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:30.217852 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96e4c69-3b4b-48f5-af08-53a818966494 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:30.219626 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96e4c69-3b4b-48f5-af08-53a818966494 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:30.244164 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 340/680] 10.222.0.97 () {56 vars in 1135 bytes} [Tue Jul 26 14:30:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:30.245525 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e4c69-3b4b-48f5-af08-53a818966494 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:30.309889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:30.352835 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8746114-619f-458b-8175-82f4546d51eb None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:30.411545 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e4c69-3b4b-48f5-af08-53a818966494 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:30.415133 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb8f909-74cd-44c9-a360-47a0695eea1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:30.415793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb8f909-74cd-44c9-a360-47a0695eea1c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:30.416076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb8f909-74cd-44c9-a360-47a0695eea1c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:30.416351 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb8f909-74cd-44c9-a360-47a0695eea1c None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:30.417054 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb8f909-74cd-44c9-a360-47a0695eea1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:30.417361 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb8f909-74cd-44c9-a360-47a0695eea1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:30.418717 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb8f909-74cd-44c9-a360-47a0695eea1c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:30.428143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:30.433272 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 341/681] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:30.434466 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb8f909-74cd-44c9-a360-47a0695eea1c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:30.474683 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f0a5d-c77c-4e8e-b97c-02c97578517a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:30.475750 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f0a5d-c77c-4e8e-b97c-02c97578517a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:30.476240 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f0a5d-c77c-4e8e-b97c-02c97578517a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:30.476646 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f0a5d-c77c-4e8e-b97c-02c97578517a None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:30.478477 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f0a5d-c77c-4e8e-b97c-02c97578517a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:30.479065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f0a5d-c77c-4e8e-b97c-02c97578517a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:30.480718 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f0a5d-c77c-4e8e-b97c-02c97578517a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:30.497503 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 341/682] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:30:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:30.498526 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f0a5d-c77c-4e8e-b97c-02c97578517a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:30.577869 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb8f909-74cd-44c9-a360-47a0695eea1c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:30.625772 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f0a5d-c77c-4e8e-b97c-02c97578517a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:30.632550 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed03108-cb48-4588-a843-d656ebe6db8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:30.633783 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed03108-cb48-4588-a843-d656ebe6db8d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:30.634020 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed03108-cb48-4588-a843-d656ebe6db8d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:30.634294 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed03108-cb48-4588-a843-d656ebe6db8d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:30.637691 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:30.647393 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed03108-cb48-4588-a843-d656ebe6db8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:30.655586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-0ed03108-cb48-4588-a843-d656ebe6db8d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:30:30.655586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:30.655586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:30.655586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:30.655586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:30.655586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:30.655586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:30.655586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:30.655586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:30.655586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:30.655586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:30.655586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:30.655586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:30.655586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:30.655586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:30.655586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:30.656552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:30.656552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:30.656552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:30.656552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:30.656552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:30.656552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:30.656552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:30.656552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:30.656552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:30.656552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:30.656552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:30.656552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:30.656552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:30.656552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:30.656552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:30.656552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:30.657360 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:30.657360 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:30.657360 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:30.657360 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:30.657360 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:30.657360 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:30.657360 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:30.657360 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:30.657360 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:30.657360 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:30.657360 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:30:30.657360 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:30.657983 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 342/683] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:30:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:30.658969 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed03108-cb48-4588-a843-d656ebe6db8d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:30.680837 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09cd7a-26a2-4bad-8d42-64ee2eb9751a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:30.681472 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09cd7a-26a2-4bad-8d42-64ee2eb9751a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:30.681756 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09cd7a-26a2-4bad-8d42-64ee2eb9751a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:30.682031 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09cd7a-26a2-4bad-8d42-64ee2eb9751a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:30.733457 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:30.799913 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db09cd7a-26a2-4bad-8d42-64ee2eb9751a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:30.801363 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db09cd7a-26a2-4bad-8d42-64ee2eb9751a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:30.807997 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed03108-cb48-4588-a843-d656ebe6db8d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:30.861856 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169a95f-56b7-47de-a4e7-c6b0f02bc457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:30.862838 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169a95f-56b7-47de-a4e7-c6b0f02bc457 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:30.863282 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169a95f-56b7-47de-a4e7-c6b0f02bc457 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:30.863792 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169a95f-56b7-47de-a4e7-c6b0f02bc457 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:30.875961 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f169a95f-56b7-47de-a4e7-c6b0f02bc457 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:30.878281 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f169a95f-56b7-47de-a4e7-c6b0f02bc457 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:30.897488 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-f169a95f-56b7-47de-a4e7-c6b0f02bc457 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:30.897488 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:30.897488 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:30.897488 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:30.897488 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:30.897488 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:30.897488 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:30.897488 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:30.897488 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:30.897488 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:30.897488 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:30.897488 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:30.897488 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:30:30.897488 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:30:30.897488 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:30:30.898440 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:30:30.898440 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:30.898440 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:30.898440 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:30.898440 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:30.898440 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:30.898440 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:30.898440 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:30.898440 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:30.898440 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:30.898440 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:30.898440 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:30.898440 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:30.898440 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:30.898440 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:30.898440 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:30.898440 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:30.899922 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:30:30.899922 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:30:30.899922 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:30:30.899922 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:30:30.899922 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:30:30.899922 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:30:30.899922 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:30.899922 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:30.907130 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 343/684] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:30:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:30.908449 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169a95f-56b7-47de-a4e7-c6b0f02bc457 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:30.923776 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 342/685] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:30:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:30:30.923776 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09cd7a-26a2-4bad-8d42-64ee2eb9751a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:30.974224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:31.002375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:31.021726 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169a95f-56b7-47de-a4e7-c6b0f02bc457 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:31.034371 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09cd7a-26a2-4bad-8d42-64ee2eb9751a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:31.076650 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1947bb9-9d44-42ae-b636-020a290f7f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:31.077583 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1947bb9-9d44-42ae-b636-020a290f7f7e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:31.078011 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1947bb9-9d44-42ae-b636-020a290f7f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:31.078410 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1947bb9-9d44-42ae-b636-020a290f7f7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:31.079624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1947bb9-9d44-42ae-b636-020a290f7f7e None admin] RBAC: Adding query filter params (role.id=9074ca9226f8483c885f32fa868a1acc, scope.project.id=c4c8da3011294783b2180ade1bfd9366, user.id=bad45b307edc415fa6185a82af64ede4) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:31.080077 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1947bb9-9d44-42ae-b636-020a290f7f7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:31.082237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1947bb9-9d44-42ae-b636-020a290f7f7e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:31.091512 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 344/686] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:30:30 2022] GET /identity/v3/role_assignments?user.id=bad45b307edc415fa6185a82af64ede4&scope.project.id=c4c8da3011294783b2180ade1bfd9366&role.id=9074ca9226f8483c885f32fa868a1acc => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:31.092723 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1947bb9-9d44-42ae-b636-020a290f7f7e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:31.101335 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f678f-d422-41f6-9c01-d0c12da1b91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:31.102366 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f678f-d422-41f6-9c01-d0c12da1b91f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:31.102645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f678f-d422-41f6-9c01-d0c12da1b91f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:31.102946 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f678f-d422-41f6-9c01-d0c12da1b91f None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:31.103709 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f678f-d422-41f6-9c01-d0c12da1b91f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:31.104023 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f678f-d422-41f6-9c01-d0c12da1b91f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:31.105404 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f678f-d422-41f6-9c01-d0c12da1b91f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:31.121338 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 343/687] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:30:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:31.122984 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f678f-d422-41f6-9c01-d0c12da1b91f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:31.279531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1947bb9-9d44-42ae-b636-020a290f7f7e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:31.300867 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:31.301671 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f678f-d422-41f6-9c01-d0c12da1b91f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:31.318847 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2bd97a-82aa-4cea-a320-f823cd6b539d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:31.320232 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2bd97a-82aa-4cea-a320-f823cd6b539d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:31.321460 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2bd97a-82aa-4cea-a320-f823cd6b539d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:31.322302 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2bd97a-82aa-4cea-a320-f823cd6b539d None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:31.324095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2bd97a-82aa-4cea-a320-f823cd6b539d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:31.324095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2bd97a-82aa-4cea-a320-f823cd6b539d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:31.326416 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2bd97a-82aa-4cea-a320-f823cd6b539d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:31.333429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:31.343633 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a01420-47f9-43df-a757-4722e4c6832f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:31.344301 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a01420-47f9-43df-a757-4722e4c6832f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:31.344592 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a01420-47f9-43df-a757-4722e4c6832f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:31.344868 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a01420-47f9-43df-a757-4722e4c6832f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:31.348585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 345/688] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:30:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:31.349854 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2bd97a-82aa-4cea-a320-f823cd6b539d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:31.362421 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a01420-47f9-43df-a757-4722e4c6832f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:31.364292 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a01420-47f9-43df-a757-4722e4c6832f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:31.381432 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-13a01420-47f9-43df-a757-4722e4c6832f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:31.381432 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:31.381432 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:31.381432 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:31.381432 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:31.381432 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:31.381432 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:31.381432 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:31.381432 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:31.381432 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:31.381432 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:31.381432 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:31.381432 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:31.381432 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:31.381432 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:31.385775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:31.385775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:31.385775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:31.385775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:31.385775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:31.385775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:31.385775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:31.385775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:31.385775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:31.385775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:31.385775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:31.385775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:31.385775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:31.385775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:31.385775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:31.385775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:31.385775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:31.387299 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:31.387299 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:31.387299 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:31.387299 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:31.387299 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 344/689] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:31.387299 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a90c3-1529-4bf7-9c7e-73e1b0384535 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:31.387299 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a90c3-1529-4bf7-9c7e-73e1b0384535 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:31.387299 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a90c3-1529-4bf7-9c7e-73e1b0384535 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:31.387299 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 345/690] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:31.389798 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a90c3-1529-4bf7-9c7e-73e1b0384535 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:31.543533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2bd97a-82aa-4cea-a320-f823cd6b539d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:31.550241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a90c3-1529-4bf7-9c7e-73e1b0384535 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:31.585852 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793fd953-0db6-497a-b488-163ef58a06b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:31.586982 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793fd953-0db6-497a-b488-163ef58a06b2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:31.587533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793fd953-0db6-497a-b488-163ef58a06b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:31.588010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793fd953-0db6-497a-b488-163ef58a06b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:31.599700 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793fd953-0db6-497a-b488-163ef58a06b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:31.600544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b1836-63c7-47a3-9db0-af5dc2652bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:31.601176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b1836-63c7-47a3-9db0-af5dc2652bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:31.601460 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b1836-63c7-47a3-9db0-af5dc2652bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:31.601739 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b1836-63c7-47a3-9db0-af5dc2652bf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:31.602605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793fd953-0db6-497a-b488-163ef58a06b2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:31.602714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b1836-63c7-47a3-9db0-af5dc2652bf0 None admin] RBAC: Adding query filter params (role.id=9074ca9226f8483c885f32fa868a1acc, scope.project.id=c4c8da3011294783b2180ade1bfd9366, user.id=c81180fb4ff04c1186a21e43fb597a91) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:31.602984 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b1836-63c7-47a3-9db0-af5dc2652bf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:31.604615 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b1836-63c7-47a3-9db0-af5dc2652bf0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:31.623485 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 346/691] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:30:31 2022] GET /identity/v3/role_assignments?user.id=c81180fb4ff04c1186a21e43fb597a91&scope.project.id=c4c8da3011294783b2180ade1bfd9366&role.id=9074ca9226f8483c885f32fa868a1acc => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:31.623485 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b1836-63c7-47a3-9db0-af5dc2652bf0 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:31.657593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-793fd953-0db6-497a-b488-163ef58a06b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:31.657593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:31.657593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:31.657593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:31.657593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:31.657593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:31.657593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:31.657593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:31.657593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:31.657593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:31.657593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:31.657593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:31.657593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:31.657593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:31.657593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:31.659038 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:31.659038 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:31.659038 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:31.659038 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:31.659038 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:31.659038 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:31.659038 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:31.659038 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:31.659038 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:31.659038 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:31.659038 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:31.659038 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:31.659038 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:31.659038 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:31.659038 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:31.659038 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:31.659038 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:31.660616 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:31.660616 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:31.660616 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:31.660616 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:31.671494 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 346/692] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:31.673874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eecee6c-75f8-4e13-ad3f-815c92a36ca8 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:31.674540 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eecee6c-75f8-4e13-ad3f-815c92a36ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:31.675127 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eecee6c-75f8-4e13-ad3f-815c92a36ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:31.718367 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-2eecee6c-75f8-4e13-ad3f-815c92a36ca8 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:31.720504 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2eecee6c-75f8-4e13-ad3f-815c92a36ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:31.763151 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:31.833798 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 347/693] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:30:31.835120 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eecee6c-75f8-4e13-ad3f-815c92a36ca8 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:31.838414 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:31.921921 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b1836-63c7-47a3-9db0-af5dc2652bf0 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:31.959171 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f279e-b3ff-4719-a53c-6b7927540813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:31.959973 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f279e-b3ff-4719-a53c-6b7927540813 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:31.960567 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f279e-b3ff-4719-a53c-6b7927540813 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:31.961019 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f279e-b3ff-4719-a53c-6b7927540813 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:31.962362 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f279e-b3ff-4719-a53c-6b7927540813 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:31.962929 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f279e-b3ff-4719-a53c-6b7927540813 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:31.965763 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f279e-b3ff-4719-a53c-6b7927540813 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:31.983633 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 347/694] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:31.984490 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f279e-b3ff-4719-a53c-6b7927540813 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:32.024431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eecee6c-75f8-4e13-ad3f-815c92a36ca8 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:32.145168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c39491-26ad-4361-a6c6-1d6ecf688b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:32.146055 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c39491-26ad-4361-a6c6-1d6ecf688b31 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:32.146491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c39491-26ad-4361-a6c6-1d6ecf688b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:32.146932 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c39491-26ad-4361-a6c6-1d6ecf688b31 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:32.147845 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c39491-26ad-4361-a6c6-1d6ecf688b31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:32.148337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c39491-26ad-4361-a6c6-1d6ecf688b31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:32.149995 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c39491-26ad-4361-a6c6-1d6ecf688b31 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:32.175907 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 348/695] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:32.177035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c39491-26ad-4361-a6c6-1d6ecf688b31 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:32.201377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f279e-b3ff-4719-a53c-6b7927540813 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:32.206901 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:32.232492 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:32.280208 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc37f9-5332-47dc-b439-0f1fb7c71fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:32.281248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc37f9-5332-47dc-b439-0f1fb7c71fcf None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:32.281770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc37f9-5332-47dc-b439-0f1fb7c71fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:32.282251 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc37f9-5332-47dc-b439-0f1fb7c71fcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:32.301785 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fc37f9-5332-47dc-b439-0f1fb7c71fcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:32.304698 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fc37f9-5332-47dc-b439-0f1fb7c71fcf None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:32.333162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-19fc37f9-5332-47dc-b439-0f1fb7c71fcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:32.333162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:32.333162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:32.333162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:32.333162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:32.333162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:32.333162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:32.333162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:32.333162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:32.333162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:32.333162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:32.333162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:32.333162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:32.333162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:32.333162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:32.334098 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:32.334098 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:32.334098 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:32.334098 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:32.334098 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:32.334098 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:32.334098 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:32.334098 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:32.334098 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:32.334098 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:32.334098 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:32.334098 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:32.334098 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:32.334098 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:32.334098 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:32.334098 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:32.334098 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:32.334963 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:32.334963 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:32.334963 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:32.334963 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:32.351749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 348/696] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:32.353542 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc37f9-5332-47dc-b439-0f1fb7c71fcf None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:32.407435 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c39491-26ad-4361-a6c6-1d6ecf688b31 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:32.528720 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc37f9-5332-47dc-b439-0f1fb7c71fcf None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:32.554528 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf8c9b-2b4f-443c-8325-4588ec0289db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:32.555959 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf8c9b-2b4f-443c-8325-4588ec0289db None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:32.556364 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf8c9b-2b4f-443c-8325-4588ec0289db None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:32.556750 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf8c9b-2b4f-443c-8325-4588ec0289db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:32.566015 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40063d-c6ff-44f9-9aa9-a087700b0a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:32.567004 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40063d-c6ff-44f9-9aa9-a087700b0a89 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:32.567542 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40063d-c6ff-44f9-9aa9-a087700b0a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:32.568026 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40063d-c6ff-44f9-9aa9-a087700b0a89 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:32.585941 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee40063d-c6ff-44f9-9aa9-a087700b0a89 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:32.588239 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee40063d-c6ff-44f9-9aa9-a087700b0a89 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:32.598106 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf8c9b-2b4f-443c-8325-4588ec0289db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:32.607436 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf8c9b-2b4f-443c-8325-4588ec0289db None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:32.608677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-ee40063d-c6ff-44f9-9aa9-a087700b0a89 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:30:32.608677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:32.608677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:32.608677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:32.608677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:32.608677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:32.608677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:32.608677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:32.608677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:32.608677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:32.608677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:32.608677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:32.608677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:30:32.608677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:30:32.608677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:30:32.608677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:30:32.609577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:32.609577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:32.609577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:30:32.609577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:32.609577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:30:32.609577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:32.609577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:32.609577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:32.609577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:32.609577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:32.609577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:32.609577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:32.609577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:32.609577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:32.609577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:32.609577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:32.609577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:32.609577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:32.610524 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:32.610524 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:32.610524 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:30:32.610524 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:30:32.610524 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:30:32.610524 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:30:32.610524 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:30:32.610524 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:30:32.610524 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:30:32.610524 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:32.618594 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 349/697] 10.222.0.97 () {56 vars in 1093 bytes} [Tue Jul 26 14:30:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:32.619871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40063d-c6ff-44f9-9aa9-a087700b0a89 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:32.648091 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:32.703363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-28cf8c9b-2b4f-443c-8325-4588ec0289db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:32.703363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:32.703363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:32.703363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:32.703363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:32.703363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:32.703363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:32.703363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:32.703363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:32.703363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:32.703363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:32.703363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:32.703363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:30:32.703363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:30:32.703363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:30:32.704626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:30:32.704626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:32.704626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:32.704626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:32.704626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:32.704626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:32.704626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:32.704626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:32.704626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:32.704626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:32.704626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:32.704626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:32.704626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:32.704626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:32.704626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:32.704626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:32.704626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:32.705487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:30:32.705487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:30:32.705487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:30:32.705487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:30:32.705487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:30:32.705487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:30:32.705487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:32.705487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:32.708072 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 349/698] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:30:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 531 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:32.713457 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf8c9b-2b4f-443c-8325-4588ec0289db None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:32.767481 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40063d-c6ff-44f9-9aa9-a087700b0a89 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:32.850065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0fa9ec-c553-40e9-807d-0fb96605b272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:32.851703 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fa9ec-c553-40e9-807d-0fb96605b272 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:32.853477 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fa9ec-c553-40e9-807d-0fb96605b272 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:32.854726 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fa9ec-c553-40e9-807d-0fb96605b272 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:32.857392 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0fa9ec-c553-40e9-807d-0fb96605b272 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:32.858307 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0fa9ec-c553-40e9-807d-0fb96605b272 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:32.863093 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0fa9ec-c553-40e9-807d-0fb96605b272 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:32.908926 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 350/699] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:32.915298 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0fa9ec-c553-40e9-807d-0fb96605b272 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:32.965807 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:33.014545 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf8c9b-2b4f-443c-8325-4588ec0289db None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:33.169971 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66799d-e546-4eac-a1cd-09be442d1c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:33.172048 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66799d-e546-4eac-a1cd-09be442d1c22 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:33.172937 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66799d-e546-4eac-a1cd-09be442d1c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:33.173624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66799d-e546-4eac-a1cd-09be442d1c22 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:33.177180 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66799d-e546-4eac-a1cd-09be442d1c22 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:33.178052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66799d-e546-4eac-a1cd-09be442d1c22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:33.182336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66799d-e546-4eac-a1cd-09be442d1c22 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:33.203142 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:33.234161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 350/700] 10.222.0.97 () {56 vars in 1133 bytes} [Tue Jul 26 14:30:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 525 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:33.235245 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66799d-e546-4eac-a1cd-09be442d1c22 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:33.279286 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0fa9ec-c553-40e9-807d-0fb96605b272 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:33.440580 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe52185-841d-4503-a358-6567086614f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:33.441496 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe52185-841d-4503-a358-6567086614f8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:33.441983 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe52185-841d-4503-a358-6567086614f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:33.442495 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe52185-841d-4503-a358-6567086614f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:33.445294 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe52185-841d-4503-a358-6567086614f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:33.446685 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe52185-841d-4503-a358-6567086614f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:33.460077 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe52185-841d-4503-a358-6567086614f8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:33.503839 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 351/701] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:30:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 594 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:33.504790 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe52185-841d-4503-a358-6567086614f8 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:33.596818 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66799d-e546-4eac-a1cd-09be442d1c22 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:33.661234 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83829ff-4785-4a29-9d41-c5adea0a9f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:33.661863 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83829ff-4785-4a29-9d41-c5adea0a9f93 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:33.662128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83829ff-4785-4a29-9d41-c5adea0a9f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:33.662413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83829ff-4785-4a29-9d41-c5adea0a9f93 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:33.663111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83829ff-4785-4a29-9d41-c5adea0a9f93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:33.665252 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83829ff-4785-4a29-9d41-c5adea0a9f93 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:33.680892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:33.723721 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe52185-841d-4503-a358-6567086614f8 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:33.763336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 351/702] 10.222.0.97 () {60 vars in 1132 bytes} [Tue Jul 26 14:30:33 2022] POST /identity/v3/users => generated 254 bytes in 528 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:30:33.776954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83829ff-4785-4a29-9d41-c5adea0a9f93 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:33.802479 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38578826-8c30-41d0-ac2f-4478e2cb0afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:33.803365 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38578826-8c30-41d0-ac2f-4478e2cb0afc None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:33.803833 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38578826-8c30-41d0-ac2f-4478e2cb0afc None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:33.804223 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38578826-8c30-41d0-ac2f-4478e2cb0afc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:33.813534 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:33.964259 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38578826-8c30-41d0-ac2f-4478e2cb0afc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:33.975439 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83829ff-4785-4a29-9d41-c5adea0a9f93 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:33.976205 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38578826-8c30-41d0-ac2f-4478e2cb0afc None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:34.025923 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155d569-5b71-47ba-bec2-6d8e69e53ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:34.025923 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155d569-5b71-47ba-bec2-6d8e69e53ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:34.026293 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155d569-5b71-47ba-bec2-6d8e69e53ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:34.026408 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155d569-5b71-47ba-bec2-6d8e69e53ee3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:34.029126 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3155d569-5b71-47ba-bec2-6d8e69e53ee3 None admin] RBAC: Adding query filter params (role.id=9074ca9226f8483c885f32fa868a1acc, scope.project.id=c4c8da3011294783b2180ade1bfd9366, user.id=85122c4052fd4ba19a2dae185a707caf) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:34.029126 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3155d569-5b71-47ba-bec2-6d8e69e53ee3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:34.032869 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3155d569-5b71-47ba-bec2-6d8e69e53ee3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:34.056079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 352/703] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:30:33 2022] GET /identity/v3/role_assignments?user.id=85122c4052fd4ba19a2dae185a707caf&scope.project.id=c4c8da3011294783b2180ade1bfd9366&role.id=9074ca9226f8483c885f32fa868a1acc => generated 608 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:34.057557 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125df2a-19a2-4a16-aac4-f96af941d853 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:34.057847 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125df2a-19a2-4a16-aac4-f96af941d853 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:34.058128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125df2a-19a2-4a16-aac4-f96af941d853 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:34.063914 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 353/704] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:34.076942 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d6ce3-fe91-4e6d-bcb7-37e6db8cc932 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:34.077225 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d6ce3-fe91-4e6d-bcb7-37e6db8cc932 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:34.077505 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d6ce3-fe91-4e6d-bcb7-37e6db8cc932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:34.128589 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ba3d6ce3-fe91-4e6d-bcb7-37e6db8cc932 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:34.129809 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ba3d6ce3-fe91-4e6d-bcb7-37e6db8cc932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:34.180792 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 352/705] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:30:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 676 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:30:34.186649 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38578826-8c30-41d0-ac2f-4478e2cb0afc None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:34.221354 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:34.225165 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:34.248762 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 354/706] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:30:34.284453 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d6ce3-fe91-4e6d-bcb7-37e6db8cc932 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:34.382700 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38578826-8c30-41d0-ac2f-4478e2cb0afc None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:34.454044 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee4e52-cede-4147-9949-ca7a666e284c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:34.456114 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee4e52-cede-4147-9949-ca7a666e284c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:34.456849 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee4e52-cede-4147-9949-ca7a666e284c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:34.457535 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee4e52-cede-4147-9949-ca7a666e284c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:34.496730 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee4e52-cede-4147-9949-ca7a666e284c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:34.500935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee4e52-cede-4147-9949-ca7a666e284c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:34.518852 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d6ce3-fe91-4e6d-bcb7-37e6db8cc932 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:34.536142 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-d4ee4e52-cede-4147-9949-ca7a666e284c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:34.536142 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:34.536142 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:34.536142 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:34.536142 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:34.536142 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:34.536142 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:34.536142 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:34.536142 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:34.536142 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:34.536142 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:34.536142 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:34.536142 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:34.536142 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:34.536142 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:34.537081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:34.537081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:34.537081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:34.537081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:34.537081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:34.537081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:34.537081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:34.537081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:34.537081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:34.537081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:34.537081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:34.537081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:34.537081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:34.537081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:34.537081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:34.537081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:34.537081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:34.537951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:34.537951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:34.537951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:34.537951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:34.538622 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 353/707] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:34.545146 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee4e52-cede-4147-9949-ca7a666e284c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:34.568774 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e59cf90-33b9-4855-9cef-fe42839cefbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:34.569674 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59cf90-33b9-4855-9cef-fe42839cefbb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:34.570128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59cf90-33b9-4855-9cef-fe42839cefbb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:34.571110 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59cf90-33b9-4855-9cef-fe42839cefbb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:34.572005 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e59cf90-33b9-4855-9cef-fe42839cefbb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:34.573530 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e59cf90-33b9-4855-9cef-fe42839cefbb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:34.585845 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-3e59cf90-33b9-4855-9cef-fe42839cefbb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 14:30:34.585845 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:34.585845 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:30:34.585845 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:30:34.585845 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:30:34.585845 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:30:34.585845 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:30:34.585845 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:30:34.585845 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 14:30:34.585845 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:34.585845 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:30:34.585845 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:34.585845 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:34.585845 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:34.585845 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:34.585845 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:34.587086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:34.587086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:34.587086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:34.587086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:34.587086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:34.587086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:34.587086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:34.587086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:30:34.587086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:30:34.587086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:30:34.587086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:30:34.587086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:34.587086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:34.587086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:34.587086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:34.587086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:34.589297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:34.589297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:34.589297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:34.589297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:34.589297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:34.589297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:34.589297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:34.589297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:34.589297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:34.589297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:34.589297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:34.589297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:30:34.589297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:30:34.589297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 14:30:34.589297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:34.592930 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 355/708] 10.222.0.97 () {56 vars in 1129 bytes} [Tue Jul 26 14:30:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:30:34.597305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430062d-b996-475a-a772-43d66890e348 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:34.598351 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430062d-b996-475a-a772-43d66890e348 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:34.599229 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430062d-b996-475a-a772-43d66890e348 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:34.603503 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 356/709] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:34 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:34.605544 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430062d-b996-475a-a772-43d66890e348 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:34.687749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:34.691618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:34.704763 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee4e52-cede-4147-9949-ca7a666e284c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:34.752697 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430062d-b996-475a-a772-43d66890e348 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:34.757439 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34699ce9-3629-48ef-b0fd-d88dc436ec73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:34.758885 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34699ce9-3629-48ef-b0fd-d88dc436ec73 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:34.759599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34699ce9-3629-48ef-b0fd-d88dc436ec73 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:34.760016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34699ce9-3629-48ef-b0fd-d88dc436ec73 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:34.760894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34699ce9-3629-48ef-b0fd-d88dc436ec73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:34.761343 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34699ce9-3629-48ef-b0fd-d88dc436ec73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:34.764136 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34699ce9-3629-48ef-b0fd-d88dc436ec73 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:34.790250 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 354/710] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:34.792557 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc046ee7-3644-4acb-b825-5cdd23b791a4 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:34.793221 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc046ee7-3644-4acb-b825-5cdd23b791a4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:34.793921 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc046ee7-3644-4acb-b825-5cdd23b791a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:34.794702 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a67ca-2715-4ecc-bcde-85861460a1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:34.795729 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a67ca-2715-4ecc-bcde-85861460a1cb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:34.796219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a67ca-2715-4ecc-bcde-85861460a1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:34.796703 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a67ca-2715-4ecc-bcde-85861460a1cb None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:34.797793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a67ca-2715-4ecc-bcde-85861460a1cb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:34.798323 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a67ca-2715-4ecc-bcde-85861460a1cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:34.800592 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a67ca-2715-4ecc-bcde-85861460a1cb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:34.809344 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 357/711] 10.222.0.97 () {56 vars in 1124 bytes} [Tue Jul 26 14:30:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:34.810765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a67ca-2715-4ecc-bcde-85861460a1cb None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:34.830641 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-cc046ee7-3644-4acb-b825-5cdd23b791a4 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:34.832254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-cc046ee7-3644-4acb-b825-5cdd23b791a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:34.947586 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 355/712] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:30:34.949438 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc046ee7-3644-4acb-b825-5cdd23b791a4 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:34.965454 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a67ca-2715-4ecc-bcde-85861460a1cb None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:34.995503 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:35.035239 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00184e-ccd6-4446-8aec-afc54a92c33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:35.036582 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00184e-ccd6-4446-8aec-afc54a92c33f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:35.037372 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00184e-ccd6-4446-8aec-afc54a92c33f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:35.038008 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00184e-ccd6-4446-8aec-afc54a92c33f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:35.061926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec00184e-ccd6-4446-8aec-afc54a92c33f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:35.063765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec00184e-ccd6-4446-8aec-afc54a92c33f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:35.074709 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:35.079756 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-ec00184e-ccd6-4446-8aec-afc54a92c33f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:35.079756 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:35.079756 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:35.079756 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:35.079756 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:35.079756 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:35.079756 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:35.079756 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:35.079756 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:35.079756 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:35.079756 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:35.079756 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:35.079756 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:30:35.079756 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:30:35.079756 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:30:35.100568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:30:35.100568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:35.100568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:35.100568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:35.100568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:35.100568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:35.100568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:35.100568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:35.100568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:35.100568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:35.100568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:35.100568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:35.100568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:35.100568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:35.100568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:35.100568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:35.100568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:35.102628 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:30:35.102628 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:30:35.102628 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:30:35.102628 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:30:35.102628 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:30:35.102628 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:30:35.102628 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:35.102628 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:35.102628 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 358/713] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:30:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:35.102628 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00184e-ccd6-4446-8aec-afc54a92c33f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:35.109232 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc046ee7-3644-4acb-b825-5cdd23b791a4 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:35.152330 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5801229-0fe8-4088-ba8c-67a1f37b4106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:35.153656 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5801229-0fe8-4088-ba8c-67a1f37b4106 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:35.154319 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5801229-0fe8-4088-ba8c-67a1f37b4106 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:35.154890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5801229-0fe8-4088-ba8c-67a1f37b4106 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:35.156011 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5801229-0fe8-4088-ba8c-67a1f37b4106 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:35.156571 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5801229-0fe8-4088-ba8c-67a1f37b4106 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:35.162595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5801229-0fe8-4088-ba8c-67a1f37b4106 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:35.169042 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 356/714] 10.222.0.97 () {56 vars in 1124 bytes} [Tue Jul 26 14:30:34 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:35.170251 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5801229-0fe8-4088-ba8c-67a1f37b4106 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:35.220632 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00184e-ccd6-4446-8aec-afc54a92c33f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:35.274774 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b2003-3d87-4ebf-ae87-4a0f64c81ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:35.275363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b2003-3d87-4ebf-ae87-4a0f64c81ded None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:35.275679 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b2003-3d87-4ebf-ae87-4a0f64c81ded None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:35.275954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b2003-3d87-4ebf-ae87-4a0f64c81ded None admin] PATH_INFO: `/v3/services/network` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:35.276605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557b2003-3d87-4ebf-ae87-4a0f64c81ded None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:35.278141 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557b2003-3d87-4ebf-ae87-4a0f64c81ded None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:35.288738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-557b2003-3d87-4ebf-ae87-4a0f64c81ded None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 14:30:35.288738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:35.288738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:30:35.288738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:30:35.288738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:30:35.288738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:30:35.288738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:30:35.288738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:30:35.288738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 14:30:35.288738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:35.288738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:30:35.288738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:35.288738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:35.288738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:35.288738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:35.288738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:35.288738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:35.290706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:35.290706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:35.290706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:35.290706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:35.290706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:35.290706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:35.290706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:30:35.290706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:30:35.290706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:30:35.290706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:30:35.290706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:35.290706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:35.290706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:35.290706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:35.290706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:35.290706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:35.291914 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:35.291914 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:35.291914 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:35.291914 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:35.291914 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:35.291914 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:35.291914 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:35.291914 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:35.291914 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:35.291914 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:35.291914 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:30:35.291914 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:30:35.291914 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 14:30:35.291914 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:35.291914 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 359/715] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:30:35 2022] GET /identity/v3/services/network => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:35.300951 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b2003-3d87-4ebf-ae87-4a0f64c81ded None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:35.341406 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5801229-0fe8-4088-ba8c-67a1f37b4106 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:35.417912 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:35.423902 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480144be-dd8b-4c5a-b467-d35d3b47fe31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:35.425010 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480144be-dd8b-4c5a-b467-d35d3b47fe31 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:35.425494 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480144be-dd8b-4c5a-b467-d35d3b47fe31 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:35.425973 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480144be-dd8b-4c5a-b467-d35d3b47fe31 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:35.427338 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480144be-dd8b-4c5a-b467-d35d3b47fe31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:35.428373 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480144be-dd8b-4c5a-b467-d35d3b47fe31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:35.434793 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480144be-dd8b-4c5a-b467-d35d3b47fe31 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:35.455547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 357/716] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:30:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:35.469302 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480144be-dd8b-4c5a-b467-d35d3b47fe31 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:35.488975 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:35.589406 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b2003-3d87-4ebf-ae87-4a0f64c81ded None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:35.677278 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fd8c2-6f56-4835-a7a8-8e0f8bfd135c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:35.677799 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fd8c2-6f56-4835-a7a8-8e0f8bfd135c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:35.678084 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fd8c2-6f56-4835-a7a8-8e0f8bfd135c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:35.678361 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fd8c2-6f56-4835-a7a8-8e0f8bfd135c None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:35.679053 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712fd8c2-6f56-4835-a7a8-8e0f8bfd135c None admin] RBAC: Adding query filter params (name=network) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:35.679358 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712fd8c2-6f56-4835-a7a8-8e0f8bfd135c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:35.681331 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712fd8c2-6f56-4835-a7a8-8e0f8bfd135c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:35.692845 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 360/717] 10.222.0.97 () {56 vars in 1112 bytes} [Tue Jul 26 14:30:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:35.696312 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fd8c2-6f56-4835-a7a8-8e0f8bfd135c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:35.750074 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480144be-dd8b-4c5a-b467-d35d3b47fe31 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:35.865123 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93208e52-27d7-4133-8c7a-b6e9b9e42948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:35.865912 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93208e52-27d7-4133-8c7a-b6e9b9e42948 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:35.866335 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93208e52-27d7-4133-8c7a-b6e9b9e42948 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:35.871512 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93208e52-27d7-4133-8c7a-b6e9b9e42948 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:35.871512 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93208e52-27d7-4133-8c7a-b6e9b9e42948 None admin] RBAC: Adding query filter params (role.id=51b2d10dfffb4b75b186f3d8b15e2a78, scope.project.id=c4c8da3011294783b2180ade1bfd9366, user.id=266abef6cda5495b897170f92043aa03) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:35.871512 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93208e52-27d7-4133-8c7a-b6e9b9e42948 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:35.871512 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93208e52-27d7-4133-8c7a-b6e9b9e42948 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:35.889176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 358/718] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:30:35 2022] GET /identity/v3/role_assignments?user.id=266abef6cda5495b897170f92043aa03&scope.project.id=c4c8da3011294783b2180ade1bfd9366&role.id=51b2d10dfffb4b75b186f3d8b15e2a78 => generated 259 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:35.958719 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fd8c2-6f56-4835-a7a8-8e0f8bfd135c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:35.967465 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:36.018041 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c38265a-6efb-489a-b8d7-5c4cabb8268f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:36.019603 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38265a-6efb-489a-b8d7-5c4cabb8268f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:36.020011 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38265a-6efb-489a-b8d7-5c4cabb8268f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:36.020409 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38265a-6efb-489a-b8d7-5c4cabb8268f None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:36.021423 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c38265a-6efb-489a-b8d7-5c4cabb8268f None admin] RBAC: Adding query filter params (type=network) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:36.021883 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c38265a-6efb-489a-b8d7-5c4cabb8268f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:36.024303 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c38265a-6efb-489a-b8d7-5c4cabb8268f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:36.042872 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 361/719] 10.222.0.97 () {56 vars in 1112 bytes} [Tue Jul 26 14:30:35 2022] GET /identity/v3/services?type=network => generated 124 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:36.951443 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22503e3-4b86-4544-bd21-9dcc95ab2625 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:36.951443 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22503e3-4b86-4544-bd21-9dcc95ab2625 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:36.951443 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22503e3-4b86-4544-bd21-9dcc95ab2625 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:36.954909 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 359/720] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:36.959366 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b324ac1-f511-45b3-919b-25cd65fbdd40 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:36.960224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b324ac1-f511-45b3-919b-25cd65fbdd40 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:36.960710 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b324ac1-f511-45b3-919b-25cd65fbdd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:37.000670 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0b324ac1-f511-45b3-919b-25cd65fbdd40 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:37.001853 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0b324ac1-f511-45b3-919b-25cd65fbdd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:37.130805 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 362/721] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:30:37.185281 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22503e3-4b86-4544-bd21-9dcc95ab2625 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:37.343868 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:37.451559 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174530f-4b97-4598-aec8-af856c6a78ae None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:37.452229 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174530f-4b97-4598-aec8-af856c6a78ae None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:37.452722 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174530f-4b97-4598-aec8-af856c6a78ae None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:37.454634 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 363/722] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:30:37.457798 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22503e3-4b86-4544-bd21-9dcc95ab2625 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:37.460291 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0d0e3-6c27-47f4-b6e0-c7ecf671e23d None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:37.461045 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0d0e3-6c27-47f4-b6e0-c7ecf671e23d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:37.461530 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0d0e3-6c27-47f4-b6e0-c7ecf671e23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:37.481032 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d3b0d0e3-6c27-47f4-b6e0-c7ecf671e23d None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:37.482576 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d3b0d0e3-6c27-47f4-b6e0-c7ecf671e23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:37.522753 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2a806f-e9ed-493e-ac19-a46ce3765ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:37.524201 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a806f-e9ed-493e-ac19-a46ce3765ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:37.524910 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a806f-e9ed-493e-ac19-a46ce3765ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:37.525501 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a806f-e9ed-493e-ac19-a46ce3765ab7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:37.543861 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2a806f-e9ed-493e-ac19-a46ce3765ab7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:37.553396 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-0b2a806f-e9ed-493e-ac19-a46ce3765ab7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:37.553396 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:37.553396 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:37.553396 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:37.553396 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:37.553396 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:37.553396 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:37.553396 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:37.553396 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:37.553396 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:37.553396 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:37.553396 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:37.553396 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:37.553396 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:37.553396 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:37.553396 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:37.554819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:37.554819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:37.554819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:37.554819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:37.554819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:37.554819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:37.554819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:37.554819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:37.554819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:37.554819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:37.554819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:37.554819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:37.554819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:37.554819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:37.554819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:37.554819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:37.556057 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:37.556057 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:37.556057 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:37.556057 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:37.556057 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:37.556057 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:37.556057 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:37.556057 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:37.556057 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:37.556057 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:37.556057 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:37.556057 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:37.563528 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 360/723] 10.222.0.97 () {56 vars in 1102 bytes} [Tue Jul 26 14:30:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:37.567729 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2a806f-e9ed-493e-ac19-a46ce3765ab7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:37.638800 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 364/724] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:30:37.675456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0d0e3-6c27-47f4-b6e0-c7ecf671e23d None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:37.738394 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:37.789453 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2a806f-e9ed-493e-ac19-a46ce3765ab7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:37.854839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0d0e3-6c27-47f4-b6e0-c7ecf671e23d None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:37.856156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:37.870904 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa27da8-573e-4ae7-8006-08caa540c139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:37.872074 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa27da8-573e-4ae7-8006-08caa540c139 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:37.872553 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa27da8-573e-4ae7-8006-08caa540c139 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:37.872955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa27da8-573e-4ae7-8006-08caa540c139 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:37.873830 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa27da8-573e-4ae7-8006-08caa540c139 None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:37.874302 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa27da8-573e-4ae7-8006-08caa540c139 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:37.875861 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa27da8-573e-4ae7-8006-08caa540c139 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:37.886341 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 361/725] 10.222.0.97 () {56 vars in 1103 bytes} [Tue Jul 26 14:30:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:30:37.888018 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a74cd7-e7f7-45ff-a9ae-5f81d03a949e None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:37.888507 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a74cd7-e7f7-45ff-a9ae-5f81d03a949e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:37.889032 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a74cd7-e7f7-45ff-a9ae-5f81d03a949e None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:37.891296 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 362/726] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:37.895899 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a74cd7-e7f7-45ff-a9ae-5f81d03a949e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:37.905556 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3c03a-b865-4b62-b2d7-ce05968dea22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:37.906460 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3c03a-b865-4b62-b2d7-ce05968dea22 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:37.906890 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3c03a-b865-4b62-b2d7-ce05968dea22 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:37.907310 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3c03a-b865-4b62-b2d7-ce05968dea22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:37.921995 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa3c03a-b865-4b62-b2d7-ce05968dea22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:37.928089 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-bfa3c03a-b865-4b62-b2d7-ce05968dea22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:30:37.928089 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:37.928089 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:37.928089 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:37.928089 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:37.928089 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:37.928089 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:37.928089 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:37.928089 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:37.928089 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:37.928089 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:37.928089 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:37.928089 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:37.928089 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:37.928089 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:37.928089 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:37.929148 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:37.929148 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:37.929148 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:37.929148 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:37.929148 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:37.929148 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:37.929148 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:37.929148 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:37.929148 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:37.929148 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:37.929148 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:37.929148 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:37.929148 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:37.929148 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:37.929148 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:37.929148 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:37.930098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:37.930098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:37.930098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:37.930098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:37.930098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:37.930098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:37.930098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:37.930098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:37.930098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:37.930098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:37.930098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:30:37.930098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:37.933470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 365/727] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:30:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:30:37.935012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b1a72-57c7-44e8-b648-4cd9d905302c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:37.935525 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b1a72-57c7-44e8-b648-4cd9d905302c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:37.935983 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b1a72-57c7-44e8-b648-4cd9d905302c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:37.960586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-fc4b1a72-57c7-44e8-b648-4cd9d905302c None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:37.962894 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fc4b1a72-57c7-44e8-b648-4cd9d905302c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:38.047836 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a74cd7-e7f7-45ff-a9ae-5f81d03a949e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:38.075804 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 366/728] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:30:38.077207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b1a72-57c7-44e8-b648-4cd9d905302c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:38.129638 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:38.135231 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f3c87-608b-48f2-b3f7-3f696d019cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:38.136124 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f3c87-608b-48f2-b3f7-3f696d019cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:38.141797 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f3c87-608b-48f2-b3f7-3f696d019cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:38.141797 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f3c87-608b-48f2-b3f7-3f696d019cd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:38.154749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068f3c87-608b-48f2-b3f7-3f696d019cd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:38.159928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068f3c87-608b-48f2-b3f7-3f696d019cd8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:38.178996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-068f3c87-608b-48f2-b3f7-3f696d019cd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:38.178996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:38.178996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:38.178996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:38.178996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:38.178996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:38.178996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:38.178996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:38.178996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:38.178996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:38.178996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:38.178996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:38.178996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:38.178996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:38.178996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:38.188452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:38.188452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:38.188452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:38.188452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:38.188452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:38.188452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:38.188452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:38.188452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:38.188452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:38.188452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:38.188452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:38.188452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:38.188452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:38.188452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:38.188452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:38.188452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:38.188452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:38.189448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:38.189448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:38.189448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:38.189448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:38.190460 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 363/729] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:30:38.191720 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f3c87-608b-48f2-b3f7-3f696d019cd8 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:38.228952 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b1a72-57c7-44e8-b648-4cd9d905302c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:38.241773 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:38.279516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35db0c-6224-49b8-b532-2fb230e94a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:38.280263 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35db0c-6224-49b8-b532-2fb230e94a84 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:38.280612 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35db0c-6224-49b8-b532-2fb230e94a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:38.281109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35db0c-6224-49b8-b532-2fb230e94a84 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:38.282053 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e35db0c-6224-49b8-b532-2fb230e94a84 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:38.282465 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e35db0c-6224-49b8-b532-2fb230e94a84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:38.285014 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e35db0c-6224-49b8-b532-2fb230e94a84 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:38.296449 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 367/730] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:30:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:38.297738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35db0c-6224-49b8-b532-2fb230e94a84 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:38.321663 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f3c87-608b-48f2-b3f7-3f696d019cd8 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:38.360841 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26daa66f-faf5-4e2d-bfd0-bd8a17572494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:38.361861 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26daa66f-faf5-4e2d-bfd0-bd8a17572494 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:38.362360 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26daa66f-faf5-4e2d-bfd0-bd8a17572494 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:38.362931 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26daa66f-faf5-4e2d-bfd0-bd8a17572494 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:38.363965 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26daa66f-faf5-4e2d-bfd0-bd8a17572494 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:38.372236 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26daa66f-faf5-4e2d-bfd0-bd8a17572494 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:38.399784 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:38.420082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35db0c-6224-49b8-b532-2fb230e94a84 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:38.456787 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b35cf5c-3203-4795-8a3c-7d53eae952c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:38.457539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b35cf5c-3203-4795-8a3c-7d53eae952c3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:38.457895 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b35cf5c-3203-4795-8a3c-7d53eae952c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:38.458296 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b35cf5c-3203-4795-8a3c-7d53eae952c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:38.459057 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b35cf5c-3203-4795-8a3c-7d53eae952c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:38.459480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b35cf5c-3203-4795-8a3c-7d53eae952c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:38.460856 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b35cf5c-3203-4795-8a3c-7d53eae952c3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:38.473540 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 368/731] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:38.473540 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b35cf5c-3203-4795-8a3c-7d53eae952c3 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:38.507455 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 364/732] 10.222.0.97 () {60 vars in 1141 bytes} [Tue Jul 26 14:30:38 2022] POST /identity/v3/services => generated 251 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:30:38.507455 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5aeb1d-dc39-4d37-ba76-d7541733e605 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:38.507455 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5aeb1d-dc39-4d37-ba76-d7541733e605 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:38.507455 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5aeb1d-dc39-4d37-ba76-d7541733e605 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:38.512803 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 365/733] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:38.513774 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5aeb1d-dc39-4d37-ba76-d7541733e605 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:38.605122 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b35cf5c-3203-4795-8a3c-7d53eae952c3 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:38.660789 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbd54d-7862-4ba9-ad1b-3eb8a189fbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:38.661612 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbd54d-7862-4ba9-ad1b-3eb8a189fbf1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:38.661724 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbd54d-7862-4ba9-ad1b-3eb8a189fbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:38.661920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbd54d-7862-4ba9-ad1b-3eb8a189fbf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:38.671218 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:38.675269 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbd54d-7862-4ba9-ad1b-3eb8a189fbf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:38.676805 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbd54d-7862-4ba9-ad1b-3eb8a189fbf1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:38.677707 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:38.679007 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5aeb1d-dc39-4d37-ba76-d7541733e605 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:38.698539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-bdfbd54d-7862-4ba9-ad1b-3eb8a189fbf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:38.698539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:38.698539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:38.698539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:38.698539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:38.698539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:38.698539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:38.698539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:38.698539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:38.698539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:38.698539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:38.698539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:38.698539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:38.698539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:38.698539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:38.699795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:38.699795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:38.699795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:38.699795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:38.699795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:38.699795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:38.699795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:38.699795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:38.699795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:38.699795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:38.699795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:38.699795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:38.699795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:38.699795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:38.699795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:38.699795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:38.699795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:38.700876 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:38.700876 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:38.700876 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:38.700876 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:38.702364 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 369/734] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:38.704446 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c922280-82ee-4c45-a368-0c067cb31b57 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:38.704446 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c922280-82ee-4c45-a368-0c067cb31b57 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:38.704446 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c922280-82ee-4c45-a368-0c067cb31b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:38.724891 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-4c922280-82ee-4c45-a368-0c067cb31b57 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:38.726819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4c922280-82ee-4c45-a368-0c067cb31b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:38.744673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2282fb-be09-48d3-9be7-09fe103de2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:38.745991 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2282fb-be09-48d3-9be7-09fe103de2bc None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:38.746568 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2282fb-be09-48d3-9be7-09fe103de2bc None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:38.747020 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2282fb-be09-48d3-9be7-09fe103de2bc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:38.830061 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 370/735] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:38 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:30:38.831092 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c922280-82ee-4c45-a368-0c067cb31b57 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:38.880089 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2282fb-be09-48d3-9be7-09fe103de2bc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:38.881727 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2282fb-be09-48d3-9be7-09fe103de2bc None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:38.895068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-1b2282fb-be09-48d3-9be7-09fe103de2bc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 14:30:38.895068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:38.895068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:38.895068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:38.895068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:38.895068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:38.895068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:38.895068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:38.895068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:38.895068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:38.895068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:38.895068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:38.895068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:30:38.895068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:30:38.895068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:30:38.895068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:30:38.896137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:38.896137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:38.896137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:30:38.896137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:38.896137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:30:38.896137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:38.896137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:38.896137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:38.896137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:38.896137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:38.896137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:38.896137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:38.896137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:38.896137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:38.896137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:38.896137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:38.896137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:38.896137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:38.897665 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:38.897665 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:38.897665 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:30:38.897665 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:30:38.897665 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:30:38.897665 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:30:38.897665 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:30:38.897665 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:30:38.897665 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 14:30:38.897665 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:38.897665 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 366/736] 10.222.0.97 () {56 vars in 1117 bytes} [Tue Jul 26 14:30:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:38.898939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f42cd2-098e-4883-9f24-6e3404d449ae None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:38.899018 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f42cd2-098e-4883-9f24-6e3404d449ae None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:38.899347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f42cd2-098e-4883-9f24-6e3404d449ae None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:38.900862 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 367/737] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:38.901571 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f42cd2-098e-4883-9f24-6e3404d449ae None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:38.963233 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c922280-82ee-4c45-a368-0c067cb31b57 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:38.987691 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:39.006291 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d79f4-5a00-483b-b47d-696ddcdbee12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:39.007859 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d79f4-5a00-483b-b47d-696ddcdbee12 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:39.008317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d79f4-5a00-483b-b47d-696ddcdbee12 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:39.008742 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d79f4-5a00-483b-b47d-696ddcdbee12 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:39.009955 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d79f4-5a00-483b-b47d-696ddcdbee12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:39.010452 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d79f4-5a00-483b-b47d-696ddcdbee12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:39.012990 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d79f4-5a00-483b-b47d-696ddcdbee12 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:39.035275 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f42cd2-098e-4883-9f24-6e3404d449ae None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:39.041892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 371/738] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:39.043872 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d79f4-5a00-483b-b47d-696ddcdbee12 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:39.097590 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7bc18e-31c9-49b5-b005-1007b6069eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:39.098315 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bc18e-31c9-49b5-b005-1007b6069eda None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:39.098735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bc18e-31c9-49b5-b005-1007b6069eda None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:39.099021 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bc18e-31c9-49b5-b005-1007b6069eda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:39.117463 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7bc18e-31c9-49b5-b005-1007b6069eda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:39.122605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-7d7bc18e-31c9-49b5-b005-1007b6069eda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:30:39.122605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:39.122605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:39.122605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:39.122605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:39.122605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:39.122605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:39.122605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:39.122605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:39.122605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:39.122605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:39.122605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:39.122605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:39.122605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:39.122605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:39.122605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:39.123603 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:39.123603 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:39.123603 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:39.123603 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:39.123603 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:39.123603 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:39.123603 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:39.123603 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:39.123603 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:39.123603 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:39.123603 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:39.123603 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:39.123603 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:39.123603 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:39.123603 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:39.123603 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:39.124567 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:39.124567 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:39.124567 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:39.124567 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:39.124567 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:39.124567 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:39.124567 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:39.124567 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:39.124567 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:39.124567 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:39.124567 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:30:39.124567 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:39.124567 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 368/739] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:30:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:39.125846 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa5781-a14a-4339-9a89-9e18a09301e7 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:39.125846 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa5781-a14a-4339-9a89-9e18a09301e7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:39.126049 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa5781-a14a-4339-9a89-9e18a09301e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:39.148636 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-63aa5781-a14a-4339-9a89-9e18a09301e7 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:39.149839 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-63aa5781-a14a-4339-9a89-9e18a09301e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:39.168138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:39.201288 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d79f4-5a00-483b-b47d-696ddcdbee12 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:39.250622 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac060e-4a82-43d8-bdf2-79331f81a9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:39.251563 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac060e-4a82-43d8-bdf2-79331f81a9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:39.251982 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac060e-4a82-43d8-bdf2-79331f81a9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:39.252554 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac060e-4a82-43d8-bdf2-79331f81a9a3 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:39.253871 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac060e-4a82-43d8-bdf2-79331f81a9a3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:39.254383 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac060e-4a82-43d8-bdf2-79331f81a9a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:39.256742 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac060e-4a82-43d8-bdf2-79331f81a9a3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:39.274932 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 369/740] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:39 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:30:39.276265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa5781-a14a-4339-9a89-9e18a09301e7 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:39.291909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 372/741] 10.222.0.97 () {56 vars in 1149 bytes} [Tue Jul 26 14:30:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:39.291909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac060e-4a82-43d8-bdf2-79331f81a9a3 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:39.344048 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:39.424616 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac060e-4a82-43d8-bdf2-79331f81a9a3 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:39.430390 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa5781-a14a-4339-9a89-9e18a09301e7 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:39.471450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a10235-c5e6-416c-822d-0d787d2dfbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:39.472080 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a10235-c5e6-416c-822d-0d787d2dfbc6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:39.472362 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a10235-c5e6-416c-822d-0d787d2dfbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:39.472635 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a10235-c5e6-416c-822d-0d787d2dfbc6 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:39.473398 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a10235-c5e6-416c-822d-0d787d2dfbc6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:39.473789 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a10235-c5e6-416c-822d-0d787d2dfbc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:39.473988 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07b6d4-4ef8-419d-aad1-f376449dcf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:39.474690 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07b6d4-4ef8-419d-aad1-f376449dcf2b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:39.475021 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07b6d4-4ef8-419d-aad1-f376449dcf2b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:39.475321 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07b6d4-4ef8-419d-aad1-f376449dcf2b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:39.475459 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a10235-c5e6-416c-822d-0d787d2dfbc6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:39.485487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 373/742] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:30:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:39.485689 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07b6d4-4ef8-419d-aad1-f376449dcf2b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:39.487267 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07b6d4-4ef8-419d-aad1-f376449dcf2b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:39.490914 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a10235-c5e6-416c-822d-0d787d2dfbc6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:39.501888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-2b07b6d4-4ef8-419d-aad1-f376449dcf2b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 14:30:39.501888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:39.501888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:39.501888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:39.501888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:39.501888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:39.501888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:39.501888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:39.501888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:39.501888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:39.501888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:39.501888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:39.501888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:30:39.501888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:30:39.501888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:30:39.501888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:30:39.503457 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:39.503457 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:39.503457 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:30:39.503457 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:39.503457 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:30:39.503457 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:39.503457 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:39.503457 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:39.503457 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:39.503457 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:39.503457 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:39.503457 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:39.503457 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:39.503457 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:39.503457 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:39.503457 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:39.503457 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:39.503457 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:39.505660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:39.505660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:39.505660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:30:39.505660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:30:39.505660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:30:39.505660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:30:39.505660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:30:39.505660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:30:39.505660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 14:30:39.505660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:39.505660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 370/743] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:30:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:39.505660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07b6d4-4ef8-419d-aad1-f376449dcf2b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:39.518635 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:39.639142 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:39.642544 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a10235-c5e6-416c-822d-0d787d2dfbc6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:39.648915 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07b6d4-4ef8-419d-aad1-f376449dcf2b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:39.699208 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e3464-6171-409d-9389-ff0436e91f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:39.703121 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e3464-6171-409d-9389-ff0436e91f47 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:39.703394 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e3464-6171-409d-9389-ff0436e91f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:39.703974 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e3464-6171-409d-9389-ff0436e91f47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:39.718201 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d242e0-86d8-4620-b484-fa913a5a5c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:39.719235 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d242e0-86d8-4620-b484-fa913a5a5c2d None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:39.719744 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d242e0-86d8-4620-b484-fa913a5a5c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:39.720219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d242e0-86d8-4620-b484-fa913a5a5c2d None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:39.721529 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d242e0-86d8-4620-b484-fa913a5a5c2d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:39.723999 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d242e0-86d8-4620-b484-fa913a5a5c2d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:39.792327 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e3464-6171-409d-9389-ff0436e91f47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:39.793909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e3464-6171-409d-9389-ff0436e91f47 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:39.833556 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 371/744] 10.222.0.97 () {60 vars in 1141 bytes} [Tue Jul 26 14:30:39 2022] POST /identity/v3/services => generated 240 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:30:39.835241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d242e0-86d8-4620-b484-fa913a5a5c2d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:39.975201 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:39.982279 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 374/745] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:30:39 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:30:39.983805 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e3464-6171-409d-9389-ff0436e91f47 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:39.987643 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:40.066525 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d242e0-86d8-4620-b484-fa913a5a5c2d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:40.119308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471e16d-c4c5-4034-92f3-779087a5db7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:40.121901 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471e16d-c4c5-4034-92f3-779087a5db7f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:40.121901 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471e16d-c4c5-4034-92f3-779087a5db7f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:40.127452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471e16d-c4c5-4034-92f3-779087a5db7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:40.153090 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8471e16d-c4c5-4034-92f3-779087a5db7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:40.155822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8471e16d-c4c5-4034-92f3-779087a5db7f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:40.181157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-8471e16d-c4c5-4034-92f3-779087a5db7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:40.181157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:40.181157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:40.181157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:40.181157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:40.181157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:40.181157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:40.181157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:40.181157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:40.181157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:40.181157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:40.181157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:40.181157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:40.181157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:40.181157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:40.182050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:40.182050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:40.182050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:40.182050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:40.182050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:40.182050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:40.182050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:40.182050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:40.182050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:40.182050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:40.182050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:40.182050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:40.182050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:40.182050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:40.182050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:40.182050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:40.182050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:40.182910 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:40.182910 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:40.182910 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:40.182910 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:40.184332 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 372/746] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:40.186006 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471e16d-c4c5-4034-92f3-779087a5db7f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:40.209226 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e3464-6171-409d-9389-ff0436e91f47 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:40.281078 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471db15-50c5-4d04-b6d5-34f3c4f1a990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:40.281889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471db15-50c5-4d04-b6d5-34f3c4f1a990 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:40.282219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471db15-50c5-4d04-b6d5-34f3c4f1a990 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:40.287441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471db15-50c5-4d04-b6d5-34f3c4f1a990 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:40.287441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471db15-50c5-4d04-b6d5-34f3c4f1a990 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:40.287441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471db15-50c5-4d04-b6d5-34f3c4f1a990 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:40.287441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471db15-50c5-4d04-b6d5-34f3c4f1a990 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:40.319264 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 375/747] 10.222.0.97 () {56 vars in 1135 bytes} [Tue Jul 26 14:30:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:40.320273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471db15-50c5-4d04-b6d5-34f3c4f1a990 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:40.397750 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471e16d-c4c5-4034-92f3-779087a5db7f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:40.404617 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:40.431061 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:40.461844 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656c166a-6fe8-4ad8-8e43-58c48ddc2875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:40.463243 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c166a-6fe8-4ad8-8e43-58c48ddc2875 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:40.464105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c166a-6fe8-4ad8-8e43-58c48ddc2875 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:40.464628 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c166a-6fe8-4ad8-8e43-58c48ddc2875 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:40.469280 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471db15-50c5-4d04-b6d5-34f3c4f1a990 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:40.487576 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656c166a-6fe8-4ad8-8e43-58c48ddc2875 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:40.490093 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656c166a-6fe8-4ad8-8e43-58c48ddc2875 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:40.520314 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-656c166a-6fe8-4ad8-8e43-58c48ddc2875 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:40.520314 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:40.520314 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:40.520314 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:40.520314 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:40.520314 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:40.520314 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:40.520314 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:40.520314 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:40.520314 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:40.520314 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:40.520314 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:40.520314 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:40.520314 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:40.520314 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:40.521951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:40.521951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:40.521951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:40.521951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:40.521951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:40.521951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:40.521951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:40.521951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:40.521951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:40.521951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:40.521951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:40.521951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:40.521951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:40.521951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:40.521951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:40.521951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:40.521951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:40.524611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:40.524611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:40.524611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:40.524611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:40.524611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 373/748] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:40.524611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656c166a-6fe8-4ad8-8e43-58c48ddc2875 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:40.538291 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec7fb4a-7e93-484a-83e8-2664def875d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:40.539349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7fb4a-7e93-484a-83e8-2664def875d8 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:40.539825 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7fb4a-7e93-484a-83e8-2664def875d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:40.540181 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7fb4a-7e93-484a-83e8-2664def875d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:40.541126 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec7fb4a-7e93-484a-83e8-2664def875d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:40.541492 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec7fb4a-7e93-484a-83e8-2664def875d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:40.544283 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec7fb4a-7e93-484a-83e8-2664def875d8 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:40.568625 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 376/749] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:40.570194 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec7fb4a-7e93-484a-83e8-2664def875d8 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:40.715432 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656c166a-6fe8-4ad8-8e43-58c48ddc2875 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:40.734247 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec7fb4a-7e93-484a-83e8-2664def875d8 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:40.770424 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:40.781158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1702ed-fff5-4898-85e9-0ea8df8a1aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:40.782445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1702ed-fff5-4898-85e9-0ea8df8a1aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:40.787061 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1702ed-fff5-4898-85e9-0ea8df8a1aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:40.787061 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1702ed-fff5-4898-85e9-0ea8df8a1aa7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:40.801346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53d3c0-81a6-43e1-a374-1adc827ea152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:40.802812 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53d3c0-81a6-43e1-a374-1adc827ea152 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:40.803914 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53d3c0-81a6-43e1-a374-1adc827ea152 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:40.806660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53d3c0-81a6-43e1-a374-1adc827ea152 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:40.808288 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53d3c0-81a6-43e1-a374-1adc827ea152 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:40.809319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53d3c0-81a6-43e1-a374-1adc827ea152 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:40.811608 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53d3c0-81a6-43e1-a374-1adc827ea152 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:40.837308 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 377/750] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:40.838618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53d3c0-81a6-43e1-a374-1adc827ea152 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:40.859715 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:40.908049 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1702ed-fff5-4898-85e9-0ea8df8a1aa7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:40.910476 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1702ed-fff5-4898-85e9-0ea8df8a1aa7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:40.966904 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53d3c0-81a6-43e1-a374-1adc827ea152 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:41.014684 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa230d0d-6a80-4a1b-9dfa-fe7a32597d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:41.015605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa230d0d-6a80-4a1b-9dfa-fe7a32597d13 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:41.015973 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa230d0d-6a80-4a1b-9dfa-fe7a32597d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:41.016437 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa230d0d-6a80-4a1b-9dfa-fe7a32597d13 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:41.030942 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa230d0d-6a80-4a1b-9dfa-fe7a32597d13 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:41.034237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa230d0d-6a80-4a1b-9dfa-fe7a32597d13 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:41.048642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-aa230d0d-6a80-4a1b-9dfa-fe7a32597d13 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:30:41.048642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:41.048642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:41.048642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:41.048642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:41.048642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:41.048642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:41.048642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:41.048642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:41.048642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:41.048642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:41.048642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:41.048642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:30:41.048642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:30:41.048642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:30:41.048642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:30:41.050416 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:41.050416 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:41.050416 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:30:41.050416 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:41.050416 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:30:41.050416 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:41.050416 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:41.050416 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:41.050416 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:41.050416 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:41.050416 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:41.050416 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:41.050416 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:41.050416 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:41.050416 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:41.050416 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:41.050416 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:41.050416 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:41.062791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:41.062791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:41.062791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:30:41.062791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:30:41.062791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:30:41.062791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:30:41.062791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:30:41.062791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:30:41.062791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:30:41.062791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:41.062791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 378/751] 10.222.0.97 () {56 vars in 1093 bytes} [Tue Jul 26 14:30:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:41.062791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa230d0d-6a80-4a1b-9dfa-fe7a32597d13 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:41.096791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 374/752] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:30:40 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:30:41.098096 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1702ed-fff5-4898-85e9-0ea8df8a1aa7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:41.108761 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:41.190378 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa230d0d-6a80-4a1b-9dfa-fe7a32597d13 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:41.202224 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:41.238215 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b2b24-de4c-4857-abc6-34075259aca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:41.239122 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b2b24-de4c-4857-abc6-34075259aca3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:41.239593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b2b24-de4c-4857-abc6-34075259aca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:41.240129 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b2b24-de4c-4857-abc6-34075259aca3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:41.248563 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b2b24-de4c-4857-abc6-34075259aca3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:41.249711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1702ed-fff5-4898-85e9-0ea8df8a1aa7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:41.250167 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b2b24-de4c-4857-abc6-34075259aca3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:41.262701 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-f94b2b24-de4c-4857-abc6-34075259aca3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:41.262701 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:41.262701 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:41.262701 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:41.262701 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:41.262701 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:41.262701 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:41.262701 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:41.262701 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:41.262701 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:41.262701 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:41.262701 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:41.262701 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:30:41.262701 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:30:41.262701 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:30:41.263652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:30:41.263652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:41.263652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:41.263652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:41.263652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:41.263652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:41.263652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:41.263652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:41.263652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:41.263652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:41.263652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:41.263652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:41.263652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:41.263652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:41.263652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:41.263652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:41.263652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:41.264500 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:30:41.264500 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:30:41.264500 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:30:41.264500 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:30:41.264500 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:30:41.264500 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:30:41.264500 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:41.264500 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:41.264500 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 379/753] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:30:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:41.265824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa09216-b90e-4df4-84de-3d2e95525de9 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:41.266190 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa09216-b90e-4df4-84de-3d2e95525de9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:41.266658 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa09216-b90e-4df4-84de-3d2e95525de9 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:41.271654 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 380/754] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:41.271654 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa09216-b90e-4df4-84de-3d2e95525de9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:41.300138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb347bb-c8a8-4ad5-9ec6-2711218d1195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:41.300891 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb347bb-c8a8-4ad5-9ec6-2711218d1195 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:41.301320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb347bb-c8a8-4ad5-9ec6-2711218d1195 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:41.301752 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb347bb-c8a8-4ad5-9ec6-2711218d1195 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:41.302933 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb347bb-c8a8-4ad5-9ec6-2711218d1195 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:41.303419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb347bb-c8a8-4ad5-9ec6-2711218d1195 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:41.305990 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb347bb-c8a8-4ad5-9ec6-2711218d1195 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:41.330039 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 375/755] 10.222.0.97 () {56 vars in 1133 bytes} [Tue Jul 26 14:30:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:41.331219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb347bb-c8a8-4ad5-9ec6-2711218d1195 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:41.399206 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:41.406208 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa09216-b90e-4df4-84de-3d2e95525de9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:41.454257 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e9aab-310f-420c-85aa-5bc54afb4634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:41.455550 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e9aab-310f-420c-85aa-5bc54afb4634 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:41.456066 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e9aab-310f-420c-85aa-5bc54afb4634 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:41.456647 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e9aab-310f-420c-85aa-5bc54afb4634 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:41.471311 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e9aab-310f-420c-85aa-5bc54afb4634 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:41.472848 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e9aab-310f-420c-85aa-5bc54afb4634 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:41.483350 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb347bb-c8a8-4ad5-9ec6-2711218d1195 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:41.495473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-b98e9aab-310f-420c-85aa-5bc54afb4634 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:41.495473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:41.495473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:41.495473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:41.495473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:41.495473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:41.495473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:41.495473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:41.495473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:41.495473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:41.495473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:41.495473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:41.495473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:41.495473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:41.495473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:41.496874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:41.496874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:41.496874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:41.496874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:41.496874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:41.496874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:41.496874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:41.496874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:41.496874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:41.496874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:41.496874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:41.496874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:41.496874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:41.496874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:41.496874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:41.496874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:41.496874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:41.498591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:41.498591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:41.498591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:41.498591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:41.498591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 381/756] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:41.498591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8de35-9dea-4afd-84da-8946670ca85b None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:41.498591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8de35-9dea-4afd-84da-8946670ca85b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:41.498591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8de35-9dea-4afd-84da-8946670ca85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:41.511903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-16b8de35-9dea-4afd-84da-8946670ca85b None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:41.513230 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-16b8de35-9dea-4afd-84da-8946670ca85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:41.527286 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:41.539349 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37bd94-5607-452d-a2e3-951abdea8bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:41.540701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37bd94-5607-452d-a2e3-951abdea8bfb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:41.542506 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37bd94-5607-452d-a2e3-951abdea8bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:41.542506 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37bd94-5607-452d-a2e3-951abdea8bfb None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:41.542980 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e37bd94-5607-452d-a2e3-951abdea8bfb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:41.543471 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e37bd94-5607-452d-a2e3-951abdea8bfb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:41.545653 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e37bd94-5607-452d-a2e3-951abdea8bfb None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:41.567384 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 376/757] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:30:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:41.568298 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37bd94-5607-452d-a2e3-951abdea8bfb None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:41.602141 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 382/758] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:30:41.603226 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8de35-9dea-4afd-84da-8946670ca85b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:41.687361 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37bd94-5607-452d-a2e3-951abdea8bfb None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:41.750099 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:41.758738 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d656d-e443-4b66-89a5-c7aa474ee2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:41.759630 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d656d-e443-4b66-89a5-c7aa474ee2a6 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:41.760044 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d656d-e443-4b66-89a5-c7aa474ee2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:41.760446 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d656d-e443-4b66-89a5-c7aa474ee2a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:41.773788 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d656d-e443-4b66-89a5-c7aa474ee2a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:41.776508 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d656d-e443-4b66-89a5-c7aa474ee2a6 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:41.786258 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8de35-9dea-4afd-84da-8946670ca85b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:41.796083 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-625d656d-e443-4b66-89a5-c7aa474ee2a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:41.796083 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:41.796083 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:41.796083 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:41.796083 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:41.796083 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:41.796083 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:41.796083 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:41.796083 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:41.796083 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:41.796083 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:41.796083 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:41.796083 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:41.796083 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:41.796083 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:41.797822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:41.797822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:41.797822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:41.797822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:41.797822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:41.797822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:41.797822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:41.797822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:41.797822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:41.797822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:41.797822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:41.797822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:41.797822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:41.797822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:41.797822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:41.797822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:41.797822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:41.800478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:41.800478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:41.800478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:41.800478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:41.800478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 377/759] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:41.801032 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d656d-e443-4b66-89a5-c7aa474ee2a6 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:41.846885 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230de9b-a2a8-4e95-abfd-6fefb7ba09c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:41.847830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230de9b-a2a8-4e95-abfd-6fefb7ba09c1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:41.848240 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230de9b-a2a8-4e95-abfd-6fefb7ba09c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:41.848666 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230de9b-a2a8-4e95-abfd-6fefb7ba09c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:41.849735 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230de9b-a2a8-4e95-abfd-6fefb7ba09c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:41.850227 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230de9b-a2a8-4e95-abfd-6fefb7ba09c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:41.853108 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230de9b-a2a8-4e95-abfd-6fefb7ba09c1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:41.869617 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 383/760] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:41.871726 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230de9b-a2a8-4e95-abfd-6fefb7ba09c1 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:41.890577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:41.989208 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d656d-e443-4b66-89a5-c7aa474ee2a6 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:42.033710 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230de9b-a2a8-4e95-abfd-6fefb7ba09c1 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:42.071767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eee55a-8335-47e6-ada7-f4d3fd415c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:42.072687 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eee55a-8335-47e6-ada7-f4d3fd415c57 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:42.073124 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eee55a-8335-47e6-ada7-f4d3fd415c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:42.073653 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eee55a-8335-47e6-ada7-f4d3fd415c57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:42.074791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eee55a-8335-47e6-ada7-f4d3fd415c57 None admin] RBAC: Adding query filter params (role.id=9074ca9226f8483c885f32fa868a1acc, scope.project.id=c4c8da3011294783b2180ade1bfd9366, user.id=5179c3c4eaaa41f3beb79ba7f1993478) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:42.075260 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eee55a-8335-47e6-ada7-f4d3fd415c57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:42.077071 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eee55a-8335-47e6-ada7-f4d3fd415c57 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:42.099439 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 378/761] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:30:41 2022] GET /identity/v3/role_assignments?user.id=5179c3c4eaaa41f3beb79ba7f1993478&scope.project.id=c4c8da3011294783b2180ade1bfd9366&role.id=9074ca9226f8483c885f32fa868a1acc => generated 259 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:42.099439 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eee55a-8335-47e6-ada7-f4d3fd415c57 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:42.133076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8680d-aa97-40a5-9691-2fe690c16e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:42.134351 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8680d-aa97-40a5-9691-2fe690c16e94 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:42.134935 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8680d-aa97-40a5-9691-2fe690c16e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:42.135572 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8680d-aa97-40a5-9691-2fe690c16e94 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:42.136779 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8680d-aa97-40a5-9691-2fe690c16e94 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:42.139754 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8680d-aa97-40a5-9691-2fe690c16e94 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:42.157470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-f4f8680d-aa97-40a5-9691-2fe690c16e94 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 14:30:42.157470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:42.157470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:30:42.157470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:30:42.157470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:30:42.157470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:30:42.157470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:30:42.157470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:30:42.157470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 14:30:42.157470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:42.157470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:30:42.157470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:42.157470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:42.157470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:42.157470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:42.157470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:42.162273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:42.162273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:42.162273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:42.162273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:42.162273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:42.162273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:42.162273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:42.162273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:30:42.162273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:30:42.162273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:30:42.162273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:30:42.162273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:42.162273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:42.162273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:42.162273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:42.162273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:42.164874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:42.164874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:42.164874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:42.164874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:42.164874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:42.164874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:42.164874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:42.164874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:42.164874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:42.164874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:42.164874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:42.164874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:30:42.164874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:30:42.164874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 14:30:42.164874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:42.164874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 384/762] 10.222.0.97 () {56 vars in 1129 bytes} [Tue Jul 26 14:30:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:42.166836 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8680d-aa97-40a5-9691-2fe690c16e94 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:42.178428 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:42.366941 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8680d-aa97-40a5-9691-2fe690c16e94 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:42.374015 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eee55a-8335-47e6-ada7-f4d3fd415c57 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:42.378680 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:42.410335 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696b369-2689-4704-8596-bd36f09de0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:42.411601 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696b369-2689-4704-8596-bd36f09de0e1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:42.412568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696b369-2689-4704-8596-bd36f09de0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:42.413168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696b369-2689-4704-8596-bd36f09de0e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:42.421946 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e696b369-2689-4704-8596-bd36f09de0e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:42.423769 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e696b369-2689-4704-8596-bd36f09de0e1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:42.426144 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac81ac-5901-404f-adfd-5c2ba7c444f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:42.427223 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac81ac-5901-404f-adfd-5c2ba7c444f7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:42.427562 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac81ac-5901-404f-adfd-5c2ba7c444f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:42.427838 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac81ac-5901-404f-adfd-5c2ba7c444f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:42.428542 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac81ac-5901-404f-adfd-5c2ba7c444f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:42.428849 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac81ac-5901-404f-adfd-5c2ba7c444f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:42.430485 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac81ac-5901-404f-adfd-5c2ba7c444f7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:42.438305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-e696b369-2689-4704-8596-bd36f09de0e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:42.438305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:42.438305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:42.438305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:42.438305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:42.438305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:42.438305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:42.438305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:42.438305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:42.438305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:42.438305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:42.438305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:42.438305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:30:42.438305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:30:42.438305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:30:42.439219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:30:42.439219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:42.439219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:42.439219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:42.439219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:42.439219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:42.439219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:42.439219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:42.439219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:42.439219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:42.439219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:42.439219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:42.439219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:42.439219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:42.439219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:42.439219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:42.439219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:42.440146 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:30:42.440146 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:30:42.440146 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:30:42.440146 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:30:42.440146 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:30:42.440146 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:30:42.440146 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:42.440146 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:42.442670 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 385/763] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:30:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:42.443873 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696b369-2689-4704-8596-bd36f09de0e1 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:42.447988 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 379/764] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:42.449599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e5684-8c33-402c-8c1c-2019c8a645b6 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:42.450093 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e5684-8c33-402c-8c1c-2019c8a645b6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:42.451030 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e5684-8c33-402c-8c1c-2019c8a645b6 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:42.453818 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 380/765] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:42.454766 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e5684-8c33-402c-8c1c-2019c8a645b6 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:42.530611 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:42.577065 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696b369-2689-4704-8596-bd36f09de0e1 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:42.593290 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e5684-8c33-402c-8c1c-2019c8a645b6 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:42.638438 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb641a-38a3-4f44-9071-beab5c4a5e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:42.638977 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb641a-38a3-4f44-9071-beab5c4a5e83 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:42.639267 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb641a-38a3-4f44-9071-beab5c4a5e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:42.639602 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb641a-38a3-4f44-9071-beab5c4a5e83 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:42.640560 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb641a-38a3-4f44-9071-beab5c4a5e83 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:42.640874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb641a-38a3-4f44-9071-beab5c4a5e83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:42.642297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb641a-38a3-4f44-9071-beab5c4a5e83 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:42.655490 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 386/766] 10.222.0.97 () {56 vars in 1124 bytes} [Tue Jul 26 14:30:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:42.655490 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb641a-38a3-4f44-9071-beab5c4a5e83 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:42.661870 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c24b2b-01b8-4ebc-aa6d-190d9777fd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:42.662520 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c24b2b-01b8-4ebc-aa6d-190d9777fd49 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:42.662812 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c24b2b-01b8-4ebc-aa6d-190d9777fd49 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:42.663280 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c24b2b-01b8-4ebc-aa6d-190d9777fd49 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:42.664702 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c24b2b-01b8-4ebc-aa6d-190d9777fd49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:42.665222 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c24b2b-01b8-4ebc-aa6d-190d9777fd49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:42.667973 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c24b2b-01b8-4ebc-aa6d-190d9777fd49 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:42.684925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 381/767] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:30:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:42.686781 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3cadad-44b0-4ba0-bf64-892dd5d27db0 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:42.686781 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3cadad-44b0-4ba0-bf64-892dd5d27db0 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:42.686781 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3cadad-44b0-4ba0-bf64-892dd5d27db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:42.706129 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-8d3cadad-44b0-4ba0-bf64-892dd5d27db0 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:42.707332 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8d3cadad-44b0-4ba0-bf64-892dd5d27db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:42.730421 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:42.795477 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 382/768] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:30:42.795477 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3cadad-44b0-4ba0-bf64-892dd5d27db0 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:42.800846 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb641a-38a3-4f44-9071-beab5c4a5e83 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:42.853336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:42.866795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b020466-1bb0-4d8a-a962-62092e385f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:42.867913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b020466-1bb0-4d8a-a962-62092e385f71 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:42.868421 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b020466-1bb0-4d8a-a962-62092e385f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:42.869082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b020466-1bb0-4d8a-a962-62092e385f71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:42.879687 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b020466-1bb0-4d8a-a962-62092e385f71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:42.881286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b020466-1bb0-4d8a-a962-62092e385f71 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:42.896939 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-1b020466-1bb0-4d8a-a962-62092e385f71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:42.896939 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:42.896939 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:42.896939 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:42.896939 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:42.896939 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:42.896939 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:42.896939 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:42.896939 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:42.896939 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:42.896939 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:42.896939 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:42.896939 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:30:42.896939 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:30:42.896939 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:30:42.898076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:30:42.898076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:42.898076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:42.898076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:42.898076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:42.898076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:42.898076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:42.898076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:42.898076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:42.898076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:42.898076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:42.898076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:42.898076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:42.898076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:42.898076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:42.898076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:42.898076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:42.898995 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:30:42.898995 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:30:42.898995 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:30:42.898995 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:30:42.898995 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:30:42.898995 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:30:42.898995 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:42.898995 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:42.898995 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 387/769] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:30:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:42.899696 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b020466-1bb0-4d8a-a962-62092e385f71 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:42.969435 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3cadad-44b0-4ba0-bf64-892dd5d27db0 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:43.029064 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999cab6-624b-4949-9c54-62365cf44db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:43.030003 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999cab6-624b-4949-9c54-62365cf44db0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:43.030459 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999cab6-624b-4949-9c54-62365cf44db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:43.030907 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999cab6-624b-4949-9c54-62365cf44db0 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:43.031821 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999cab6-624b-4949-9c54-62365cf44db0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:43.032750 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999cab6-624b-4949-9c54-62365cf44db0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:43.034701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999cab6-624b-4949-9c54-62365cf44db0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:43.042249 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 383/770] 10.222.0.97 () {56 vars in 1124 bytes} [Tue Jul 26 14:30:42 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:43.043317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999cab6-624b-4949-9c54-62365cf44db0 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:43.061267 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b020466-1bb0-4d8a-a962-62092e385f71 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:43.076138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:43.114208 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffbcb12-7917-45a1-b090-c230e845b510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:43.115093 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffbcb12-7917-45a1-b090-c230e845b510 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:43.116158 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffbcb12-7917-45a1-b090-c230e845b510 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:43.116651 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffbcb12-7917-45a1-b090-c230e845b510 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:43.117604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffbcb12-7917-45a1-b090-c230e845b510 None admin] RBAC: Adding query filter params (role.id=51b2d10dfffb4b75b186f3d8b15e2a78, scope.project.id=c4c8da3011294783b2180ade1bfd9366, user.id=85122c4052fd4ba19a2dae185a707caf) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:43.118483 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffbcb12-7917-45a1-b090-c230e845b510 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:43.121483 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffbcb12-7917-45a1-b090-c230e845b510 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:43.128449 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 388/771] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:30:42 2022] GET /identity/v3/role_assignments?user.id=85122c4052fd4ba19a2dae185a707caf&scope.project.id=c4c8da3011294783b2180ade1bfd9366&role.id=51b2d10dfffb4b75b186f3d8b15e2a78 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:43.130037 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffbcb12-7917-45a1-b090-c230e845b510 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:43.239264 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:43.282135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999cab6-624b-4949-9c54-62365cf44db0 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:43.326355 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffbcb12-7917-45a1-b090-c230e845b510 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:43.377727 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954f2e3-8522-42d9-b82c-ec8100663051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:43.385825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954f2e3-8522-42d9-b82c-ec8100663051 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:43.386692 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954f2e3-8522-42d9-b82c-ec8100663051 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:43.387343 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954f2e3-8522-42d9-b82c-ec8100663051 None admin] PATH_INFO: `/v3/services/network` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:43.388729 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954f2e3-8522-42d9-b82c-ec8100663051 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:43.391631 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954f2e3-8522-42d9-b82c-ec8100663051 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:43.405914 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d542d-910f-4cbc-8283-29b61daa30c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:43.406559 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d542d-910f-4cbc-8283-29b61daa30c1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:43.406845 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d542d-910f-4cbc-8283-29b61daa30c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:43.407125 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d542d-910f-4cbc-8283-29b61daa30c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:43.407925 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d542d-910f-4cbc-8283-29b61daa30c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:43.408238 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d542d-910f-4cbc-8283-29b61daa30c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:43.409974 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d542d-910f-4cbc-8283-29b61daa30c1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:43.414147 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-4954f2e3-8522-42d9-b82c-ec8100663051 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 14:30:43.414147 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:43.414147 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:30:43.414147 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:30:43.414147 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:30:43.414147 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:30:43.414147 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:30:43.414147 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:30:43.414147 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 14:30:43.414147 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:43.414147 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:30:43.414147 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:43.414147 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:43.414147 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:43.414147 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:43.414147 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:43.414147 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:43.416123 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:43.416123 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:43.416123 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:43.416123 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:43.416123 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:43.416123 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:43.416123 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:30:43.416123 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:30:43.416123 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:30:43.416123 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:30:43.416123 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:43.416123 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:43.416123 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:43.416123 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:43.416123 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:43.416123 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:43.417544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:43.417544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:43.417544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:43.417544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:43.417544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:43.417544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:43.417544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:43.417544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:43.417544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:43.417544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:43.417544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:30:43.417544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:30:43.417544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 14:30:43.417544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:43.421367 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 384/772] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:30:43 2022] GET /identity/v3/services/network => generated 88 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:43.422884 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954f2e3-8522-42d9-b82c-ec8100663051 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:43.439607 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 389/773] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:30:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:43.441231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d542d-910f-4cbc-8283-29b61daa30c1 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:43.592062 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d542d-910f-4cbc-8283-29b61daa30c1 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:43.608640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:43.611888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:43.629946 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954f2e3-8522-42d9-b82c-ec8100663051 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:43.653235 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb7411-9560-4190-be9b-c9796ceea8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:43.653907 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb7411-9560-4190-be9b-c9796ceea8a2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:43.654191 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb7411-9560-4190-be9b-c9796ceea8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:43.654470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb7411-9560-4190-be9b-c9796ceea8a2 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:43.655231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb7411-9560-4190-be9b-c9796ceea8a2 None admin] RBAC: Adding query filter params (name=network) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:43.655601 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb7411-9560-4190-be9b-c9796ceea8a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:43.657051 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb7411-9560-4190-be9b-c9796ceea8a2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:43.676523 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 390/774] 10.222.0.97 () {56 vars in 1112 bytes} [Tue Jul 26 14:30:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:43.677504 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb7411-9560-4190-be9b-c9796ceea8a2 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:43.687858 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa76db3-6704-4ccf-82a8-c152c90c8c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:43.689174 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa76db3-6704-4ccf-82a8-c152c90c8c52 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:43.689781 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa76db3-6704-4ccf-82a8-c152c90c8c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:43.690410 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa76db3-6704-4ccf-82a8-c152c90c8c52 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:43.691789 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa76db3-6704-4ccf-82a8-c152c90c8c52 None admin] RBAC: Adding query filter params (interface=public, service_id=b0b18a3241f8417aab2336e0540a04e9, region_id=RegionOne) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:43.692449 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa76db3-6704-4ccf-82a8-c152c90c8c52 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:43.694564 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa76db3-6704-4ccf-82a8-c152c90c8c52 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:43.709105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 385/775] 10.222.0.97 () {56 vars in 1251 bytes} [Tue Jul 26 14:30:43 2022] GET /identity/v3/endpoints?service_id=b0b18a3241f8417aab2336e0540a04e9&interface=public®ion_id=RegionOne => generated 194 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:43.710882 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa76db3-6704-4ccf-82a8-c152c90c8c52 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:43.855154 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb7411-9560-4190-be9b-c9796ceea8a2 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:43.915455 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa76db3-6704-4ccf-82a8-c152c90c8c52 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:43.916908 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f53911-1cd3-4d94-af56-e154f44dcfb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:43.917845 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f53911-1cd3-4d94-af56-e154f44dcfb6 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:43.918337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f53911-1cd3-4d94-af56-e154f44dcfb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:43.918822 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f53911-1cd3-4d94-af56-e154f44dcfb6 None admin] PATH_INFO: `/v3/projects/c4c8da3011294783b2180ade1bfd9366/users/266abef6cda5495b897170f92043aa03/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:43.922267 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f53911-1cd3-4d94-af56-e154f44dcfb6 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4c8da3011294783b2180ade1bfd9366, user_id=266abef6cda5495b897170f92043aa03, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:43.925067 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f53911-1cd3-4d94-af56-e154f44dcfb6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:43.987049 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a478b9-43f1-484c-a2c5-8ffa663e4ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:43.988079 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a478b9-43f1-484c-a2c5-8ffa663e4ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:43.988524 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a478b9-43f1-484c-a2c5-8ffa663e4ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:43.988959 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a478b9-43f1-484c-a2c5-8ffa663e4ad0 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:43.990172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a478b9-43f1-484c-a2c5-8ffa663e4ad0 None admin] RBAC: Adding query filter params (type=network) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:43.997605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a478b9-43f1-484c-a2c5-8ffa663e4ad0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:43.997605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a478b9-43f1-484c-a2c5-8ffa663e4ad0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:44.011940 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 386/776] 10.222.0.97 () {56 vars in 1112 bytes} [Tue Jul 26 14:30:43 2022] GET /identity/v3/services?type=network => generated 350 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:44.013830 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a478b9-43f1-484c-a2c5-8ffa663e4ad0 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:44.035975 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 391/777] 10.222.0.97 () {58 vars in 1439 bytes} [Tue Jul 26 14:30:43 2022] PUT /identity/v3/projects/c4c8da3011294783b2180ade1bfd9366/users/266abef6cda5495b897170f92043aa03/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:30:44.043079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f53911-1cd3-4d94-af56-e154f44dcfb6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:44.062637 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:44.219015 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:44.426610 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f53911-1cd3-4d94-af56-e154f44dcfb6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:44.432008 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a478b9-43f1-484c-a2c5-8ffa663e4ad0 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:44.525094 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ee47a-5bb3-48fd-968a-38ee1118ce2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:44.526039 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ee47a-5bb3-48fd-968a-38ee1118ce2a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:44.526465 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ee47a-5bb3-48fd-968a-38ee1118ce2a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:44.526886 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ee47a-5bb3-48fd-968a-38ee1118ce2a None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:44.527949 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ee47a-5bb3-48fd-968a-38ee1118ce2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:44.530224 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ee47a-5bb3-48fd-968a-38ee1118ce2a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:44.540986 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 387/778] 10.222.0.97 () {56 vars in 1087 bytes} [Tue Jul 26 14:30:44 2022] GET /identity/v3/services => generated 772 bytes in 528 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:44.592632 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ca3496-3fff-4c4c-bf7b-72c10fe2d44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:44.594260 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca3496-3fff-4c4c-bf7b-72c10fe2d44d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:44.595246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca3496-3fff-4c4c-bf7b-72c10fe2d44d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:44.595919 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca3496-3fff-4c4c-bf7b-72c10fe2d44d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:44.597344 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ca3496-3fff-4c4c-bf7b-72c10fe2d44d None admin] RBAC: Adding query filter params (interface=public, service_id=1cc0fe9d099f4bda96a0f36d17069e98, region_id=RegionOne) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:44.598041 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ca3496-3fff-4c4c-bf7b-72c10fe2d44d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:44.600367 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ca3496-3fff-4c4c-bf7b-72c10fe2d44d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:44.630289 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 392/779] 10.222.0.97 () {56 vars in 1251 bytes} [Tue Jul 26 14:30:44 2022] GET /identity/v3/endpoints?service_id=1cc0fe9d099f4bda96a0f36d17069e98&interface=public®ion_id=RegionOne => generated 194 bytes in 594 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:44.639218 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ee47a-5bb3-48fd-968a-38ee1118ce2a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:44.839673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:44.881167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ee47a-5bb3-48fd-968a-38ee1118ce2a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:44.943690 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624b88d-52dc-4bb7-8dda-b35bdcd0a478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:44.945211 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624b88d-52dc-4bb7-8dda-b35bdcd0a478 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:44.945642 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624b88d-52dc-4bb7-8dda-b35bdcd0a478 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:44.946043 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624b88d-52dc-4bb7-8dda-b35bdcd0a478 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:44.947111 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624b88d-52dc-4bb7-8dda-b35bdcd0a478 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:44.949558 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624b88d-52dc-4bb7-8dda-b35bdcd0a478 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:44.959816 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 388/780] 10.222.0.97 () {56 vars in 1087 bytes} [Tue Jul 26 14:30:44 2022] GET /identity/v3/services => generated 772 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:45.091783 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ecf8b-9e7d-4b68-8b33-84003c7cd5f1 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:45.092111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ecf8b-9e7d-4b68-8b33-84003c7cd5f1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:45.092311 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ecf8b-9e7d-4b68-8b33-84003c7cd5f1 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:45.094038 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 393/781] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:45.100203 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353fbd8-7da6-4181-af02-a4e77846d3c5 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:45.100506 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353fbd8-7da6-4181-af02-a4e77846d3c5 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:45.101375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353fbd8-7da6-4181-af02-a4e77846d3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:45.143343 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c353fbd8-7da6-4181-af02-a4e77846d3c5 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:45.145464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c353fbd8-7da6-4181-af02-a4e77846d3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:45.296270 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 389/782] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:30:45.344166 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ecf8b-9e7d-4b68-8b33-84003c7cd5f1 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:45.564037 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:45.566472 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ecf8b-9e7d-4b68-8b33-84003c7cd5f1 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:45.622041 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea936ae-5afa-4a35-9fdd-37703d9e123f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:45.622612 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea936ae-5afa-4a35-9fdd-37703d9e123f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:45.622895 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea936ae-5afa-4a35-9fdd-37703d9e123f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:45.623170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea936ae-5afa-4a35-9fdd-37703d9e123f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:45.642073 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea936ae-5afa-4a35-9fdd-37703d9e123f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:45.655529 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-4ea936ae-5afa-4a35-9fdd-37703d9e123f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:45.655529 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:45.655529 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:45.655529 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:45.655529 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:45.655529 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:45.655529 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:45.655529 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:45.655529 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:45.655529 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:45.655529 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:45.655529 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:45.655529 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:45.655529 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:45.655529 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:45.655529 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:45.656949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:45.656949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:45.656949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:45.656949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:45.656949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:45.656949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:45.656949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:45.656949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:45.656949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:45.656949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:45.656949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:45.656949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:45.656949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:45.656949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:45.656949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:45.656949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:45.658156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:45.658156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:45.658156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:45.658156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:45.658156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:45.658156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:45.658156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:45.658156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:45.658156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:45.658156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:45.658156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:45.658156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:45.661006 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 394/783] 10.222.0.97 () {56 vars in 1102 bytes} [Tue Jul 26 14:30:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:45.668335 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353fbd8-7da6-4181-af02-a4e77846d3c5 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:45.812737 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:45.877601 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353fbd8-7da6-4181-af02-a4e77846d3c5 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:45.952146 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd073a-d3bf-4784-88e0-0ffa9618edf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:45.952783 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd073a-d3bf-4784-88e0-0ffa9618edf3 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:45.953068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd073a-d3bf-4784-88e0-0ffa9618edf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:45.953344 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd073a-d3bf-4784-88e0-0ffa9618edf3 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:45.954079 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd073a-d3bf-4784-88e0-0ffa9618edf3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:45.954396 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd073a-d3bf-4784-88e0-0ffa9618edf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:45.955790 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd073a-d3bf-4784-88e0-0ffa9618edf3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:45.970319 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 390/784] 10.222.0.97 () {56 vars in 1103 bytes} [Tue Jul 26 14:30:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:45.978537 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea936ae-5afa-4a35-9fdd-37703d9e123f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:46.102046 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3709e0-4539-44b9-b167-e6e3cb83e68e None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:46.102427 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3709e0-4539-44b9-b167-e6e3cb83e68e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:46.102789 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3709e0-4539-44b9-b167-e6e3cb83e68e None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:46.106953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 391/785] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:46.117807 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecac08-7496-4114-b151-124a5e630de0 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:46.118895 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecac08-7496-4114-b151-124a5e630de0 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:46.119171 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecac08-7496-4114-b151-124a5e630de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:46.155332 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-84ecac08-7496-4114-b151-124a5e630de0 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:46.158088 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-84ecac08-7496-4114-b151-124a5e630de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:46.174893 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea936ae-5afa-4a35-9fdd-37703d9e123f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:46.243877 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900fd322-423f-4db9-80a2-1958a04ee669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:46.244580 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fd322-423f-4db9-80a2-1958a04ee669 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:46.244864 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fd322-423f-4db9-80a2-1958a04ee669 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:46.245136 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fd322-423f-4db9-80a2-1958a04ee669 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:46.257530 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900fd322-423f-4db9-80a2-1958a04ee669 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:46.259000 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900fd322-423f-4db9-80a2-1958a04ee669 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:46.280188 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-900fd322-423f-4db9-80a2-1958a04ee669 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:46.280188 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:46.280188 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:46.280188 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:46.280188 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:46.280188 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:46.280188 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:46.280188 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:46.280188 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:46.280188 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:46.280188 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:46.280188 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:46.280188 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:46.280188 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:46.280188 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:46.281128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:46.281128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:46.281128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:46.281128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:46.281128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:46.281128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:46.281128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:46.281128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:46.281128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:46.281128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:46.281128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:46.281128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:46.281128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:46.281128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:46.281128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:46.281128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:46.281128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:46.282005 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:46.282005 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:46.282005 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:46.282005 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:46.286983 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 395/786] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:46.289248 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900fd322-423f-4db9-80a2-1958a04ee669 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:46.311968 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 392/787] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:30:46.323383 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:46.354774 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ecac08-7496-4114-b151-124a5e630de0 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:46.468060 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:46.522243 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900fd322-423f-4db9-80a2-1958a04ee669 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:46.548750 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ecac08-7496-4114-b151-124a5e630de0 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:46.601598 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c80a9-b44c-4dad-870a-c4612332ab3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:46.602462 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c80a9-b44c-4dad-870a-c4612332ab3c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:46.602855 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c80a9-b44c-4dad-870a-c4612332ab3c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:46.603241 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c80a9-b44c-4dad-870a-c4612332ab3c None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:46.604262 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c80a9-b44c-4dad-870a-c4612332ab3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:46.604697 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c80a9-b44c-4dad-870a-c4612332ab3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:46.606706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c80a9-b44c-4dad-870a-c4612332ab3c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:46.643203 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f927b34-0312-4e91-b9f8-acc9b778a8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:46.643886 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f927b34-0312-4e91-b9f8-acc9b778a8fc None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:46.644163 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f927b34-0312-4e91-b9f8-acc9b778a8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:46.644435 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f927b34-0312-4e91-b9f8-acc9b778a8fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:46.644928 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 396/788] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:46.647429 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c80a9-b44c-4dad-870a-c4612332ab3c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:46.671002 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f927b34-0312-4e91-b9f8-acc9b778a8fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:46.677323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-0f927b34-0312-4e91-b9f8-acc9b778a8fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:30:46.677323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:46.677323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:46.677323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:46.677323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:46.677323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:46.677323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:46.677323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:46.677323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:46.677323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:46.677323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:46.677323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:46.677323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:46.677323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:46.677323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:46.677323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:46.678279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:46.678279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:46.678279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:46.678279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:46.678279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:46.678279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:46.678279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:46.678279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:46.678279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:46.678279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:46.678279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:46.678279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:46.678279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:46.678279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:46.678279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:46.678279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:46.679701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:46.679701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:46.679701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:46.679701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:46.679701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:46.679701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:46.679701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:46.679701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:46.679701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:46.679701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:46.679701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:30:46.679701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:46.684403 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 393/789] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:30:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:46.688159 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f927b34-0312-4e91-b9f8-acc9b778a8fc None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:46.803878 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:46.843066 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c80a9-b44c-4dad-870a-c4612332ab3c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:46.910943 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6473c-03bd-4b01-875e-ee3f9f895067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:46.911637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6473c-03bd-4b01-875e-ee3f9f895067 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:46.912598 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6473c-03bd-4b01-875e-ee3f9f895067 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:46.912905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6473c-03bd-4b01-875e-ee3f9f895067 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:46.926564 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b6473c-03bd-4b01-875e-ee3f9f895067 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:46.930140 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f927b34-0312-4e91-b9f8-acc9b778a8fc None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:46.932318 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b6473c-03bd-4b01-875e-ee3f9f895067 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:46.958337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-96b6473c-03bd-4b01-875e-ee3f9f895067 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 14:30:46.958337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:46.958337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:46.958337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:46.958337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:46.958337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:46.958337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:46.958337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:46.958337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:46.958337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:46.958337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:46.958337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:46.958337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:30:46.958337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:30:46.958337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:30:46.958337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:30:46.960399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:46.960399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:46.960399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:30:46.960399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:46.960399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:30:46.960399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:46.960399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:46.960399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:46.960399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:46.960399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:46.960399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:46.960399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:46.960399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:46.960399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:46.960399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:46.960399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:46.960399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:46.960399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:46.961702 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:46.961702 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:46.961702 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:30:46.961702 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:30:46.961702 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:30:46.961702 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:30:46.961702 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:30:46.961702 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:30:46.961702 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 14:30:46.961702 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:46.961702 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 397/790] 10.222.0.97 () {56 vars in 1117 bytes} [Tue Jul 26 14:30:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:30:46.962399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24671a5-2f34-4018-b92f-4edf9bcddc1a None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:46.962486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24671a5-2f34-4018-b92f-4edf9bcddc1a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:46.962696 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24671a5-2f34-4018-b92f-4edf9bcddc1a None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:46.967515 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 398/791] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:46.971186 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24671a5-2f34-4018-b92f-4edf9bcddc1a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:46.974473 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:46.999430 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9dde87-2a58-457f-b1a1-0a907c253695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:46.999430 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9dde87-2a58-457f-b1a1-0a907c253695 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:46.999430 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9dde87-2a58-457f-b1a1-0a907c253695 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:46.999430 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9dde87-2a58-457f-b1a1-0a907c253695 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:46.999935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9dde87-2a58-457f-b1a1-0a907c253695 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:46.999935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9dde87-2a58-457f-b1a1-0a907c253695 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:47.003438 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9dde87-2a58-457f-b1a1-0a907c253695 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:47.021030 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 394/792] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:30:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:47.029632 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201dcc3e-6428-41df-8f3c-0aa8018d43ab None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:47.030150 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201dcc3e-6428-41df-8f3c-0aa8018d43ab None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:47.031450 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201dcc3e-6428-41df-8f3c-0aa8018d43ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:47.061036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-201dcc3e-6428-41df-8f3c-0aa8018d43ab None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:47.062968 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-201dcc3e-6428-41df-8f3c-0aa8018d43ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:47.118659 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24671a5-2f34-4018-b92f-4edf9bcddc1a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:47.151910 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:47.157932 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52800b25-0cab-4ad2-8aa1-d727696b68dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:47.158609 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52800b25-0cab-4ad2-8aa1-d727696b68dd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:47.159132 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52800b25-0cab-4ad2-8aa1-d727696b68dd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:47.159252 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52800b25-0cab-4ad2-8aa1-d727696b68dd None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:47.160100 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52800b25-0cab-4ad2-8aa1-d727696b68dd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:47.160308 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52800b25-0cab-4ad2-8aa1-d727696b68dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:47.161792 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52800b25-0cab-4ad2-8aa1-d727696b68dd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:47.165858 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 395/793] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:30:47.168170 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201dcc3e-6428-41df-8f3c-0aa8018d43ab None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:47.181221 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 399/794] 10.222.0.97 () {56 vars in 1149 bytes} [Tue Jul 26 14:30:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:47.192462 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52800b25-0cab-4ad2-8aa1-d727696b68dd None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:47.303062 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201dcc3e-6428-41df-8f3c-0aa8018d43ab None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:47.304976 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52800b25-0cab-4ad2-8aa1-d727696b68dd None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:47.322626 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:47.348844 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7be06-d333-4afb-bf16-ed06bb09c29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:47.349648 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7be06-d333-4afb-bf16-ed06bb09c29d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:47.350074 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7be06-d333-4afb-bf16-ed06bb09c29d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:47.350460 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7be06-d333-4afb-bf16-ed06bb09c29d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:47.350732 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5fd298-af05-461f-805c-93127e2fd8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:47.351782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fd298-af05-461f-805c-93127e2fd8ad None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:47.352240 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fd298-af05-461f-805c-93127e2fd8ad None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:47.352901 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fd298-af05-461f-805c-93127e2fd8ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:47.364142 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab7be06-d333-4afb-bf16-ed06bb09c29d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:47.368566 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab7be06-d333-4afb-bf16-ed06bb09c29d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:47.370822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5fd298-af05-461f-805c-93127e2fd8ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:47.372564 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5fd298-af05-461f-805c-93127e2fd8ad None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:47.383627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-2ab7be06-d333-4afb-bf16-ed06bb09c29d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:47.383627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:47.383627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:47.383627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:47.383627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:47.383627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:47.383627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:47.383627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:47.383627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:47.383627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:47.383627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:47.383627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:47.383627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:47.383627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:47.383627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:47.384760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:47.384760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:47.384760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:47.384760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:47.384760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:47.384760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:47.384760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:47.384760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:47.384760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:47.384760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:47.384760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:47.384760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:47.384760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:47.384760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:47.384760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:47.384760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:47.384760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:47.385698 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:47.385698 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:47.385698 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:47.385698 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:47.389835 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 400/795] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:30:47.391044 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7be06-d333-4afb-bf16-ed06bb09c29d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:47.396672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-8b5fd298-af05-461f-805c-93127e2fd8ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:47.396672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:47.396672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:47.396672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:47.396672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:47.396672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:47.396672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:47.396672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:47.396672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:47.396672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:47.396672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:47.396672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:47.396672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:47.396672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:47.396672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:47.397591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:47.397591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:47.397591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:47.397591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:47.397591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:47.397591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:47.397591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:47.397591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:47.397591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:47.397591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:47.397591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:47.397591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:47.397591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:47.397591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:47.397591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:47.397591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:47.397591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:47.398496 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:47.398496 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:47.398496 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:47.398496 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:47.405347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 396/796] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:47.405347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5fd298-af05-461f-805c-93127e2fd8ad None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:47.443330 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:47.531433 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7be06-d333-4afb-bf16-ed06bb09c29d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:47.544747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5fd298-af05-461f-805c-93127e2fd8ad None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:47.582334 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc5a8e-496b-4f9f-8cf4-4e0767e7bfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:47.583361 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc5a8e-496b-4f9f-8cf4-4e0767e7bfb9 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:47.583899 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc5a8e-496b-4f9f-8cf4-4e0767e7bfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:47.584549 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc5a8e-496b-4f9f-8cf4-4e0767e7bfb9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:47.598476 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bc5a8e-496b-4f9f-8cf4-4e0767e7bfb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:47.605941 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-b3bc5a8e-496b-4f9f-8cf4-4e0767e7bfb9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:30:47.605941 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:47.605941 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:47.605941 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:47.605941 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:47.605941 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:47.605941 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:47.605941 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:47.605941 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:47.605941 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:47.605941 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:47.605941 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:47.605941 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:47.605941 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:47.605941 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:47.605941 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:47.606920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:47.606920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:47.606920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:47.606920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:47.606920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:47.606920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:47.606920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:47.606920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:47.606920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:47.606920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:47.606920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:47.606920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:47.606920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:47.606920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:47.606920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:47.606920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:47.607803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:47.607803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:47.607803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:47.607803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:47.607803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:47.607803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:47.607803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:47.607803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:47.607803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:47.607803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:47.607803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:30:47.607803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:47.610839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 401/797] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:30:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:47.611983 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc5a8e-496b-4f9f-8cf4-4e0767e7bfb9 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:47.617066 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42886a04-4006-4d25-8b4c-b13f4c99d368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:47.617637 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42886a04-4006-4d25-8b4c-b13f4c99d368 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:47.617953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42886a04-4006-4d25-8b4c-b13f4c99d368 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:47.618228 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42886a04-4006-4d25-8b4c-b13f4c99d368 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:47.619434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42886a04-4006-4d25-8b4c-b13f4c99d368 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:47.619434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42886a04-4006-4d25-8b4c-b13f4c99d368 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:47.620754 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42886a04-4006-4d25-8b4c-b13f4c99d368 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:47.639035 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 397/798] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:47.641816 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aeebba-af10-4ba6-ad5b-12fd0b8c7e8f None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:47.642108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aeebba-af10-4ba6-ad5b-12fd0b8c7e8f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:47.642383 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aeebba-af10-4ba6-ad5b-12fd0b8c7e8f None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:47.648157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 398/799] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:47.648157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aeebba-af10-4ba6-ad5b-12fd0b8c7e8f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:47.681245 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:47.739774 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc5a8e-496b-4f9f-8cf4-4e0767e7bfb9 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:47.742297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:47.778278 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b5894-c837-44a3-9778-da0b2a194345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:47.779444 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b5894-c837-44a3-9778-da0b2a194345 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:47.779672 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b5894-c837-44a3-9778-da0b2a194345 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:47.780101 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b5894-c837-44a3-9778-da0b2a194345 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:47.781250 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b5894-c837-44a3-9778-da0b2a194345 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:47.781780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b5894-c837-44a3-9778-da0b2a194345 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:47.783685 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b5894-c837-44a3-9778-da0b2a194345 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:47.792119 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aeebba-af10-4ba6-ad5b-12fd0b8c7e8f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:47.801407 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 402/800] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:47.802731 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b5894-c837-44a3-9778-da0b2a194345 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:47.845689 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ced83-b9fb-447a-87b7-a6394fa1fe9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:47.846353 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ced83-b9fb-447a-87b7-a6394fa1fe9e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:47.846641 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ced83-b9fb-447a-87b7-a6394fa1fe9e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:47.847127 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ced83-b9fb-447a-87b7-a6394fa1fe9e None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:47.847823 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512ced83-b9fb-447a-87b7-a6394fa1fe9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:47.848137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512ced83-b9fb-447a-87b7-a6394fa1fe9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:47.849547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512ced83-b9fb-447a-87b7-a6394fa1fe9e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:47.867291 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 399/801] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:30:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:47.868900 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b61913-9b7f-43ac-933c-1dbafe3242c7 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:47.869323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b61913-9b7f-43ac-933c-1dbafe3242c7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:47.869645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b61913-9b7f-43ac-933c-1dbafe3242c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:47.892390 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c2b61913-9b7f-43ac-933c-1dbafe3242c7 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:47.893746 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c2b61913-9b7f-43ac-933c-1dbafe3242c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:47.938052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b5894-c837-44a3-9778-da0b2a194345 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:47.969620 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 400/802] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:30:47.970917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac3e52-cae9-44df-acad-ddd6b2cbd295 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:47.971214 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac3e52-cae9-44df-acad-ddd6b2cbd295 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:47.971766 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac3e52-cae9-44df-acad-ddd6b2cbd295 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:47.977219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 401/803] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:47.977219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac3e52-cae9-44df-acad-ddd6b2cbd295 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:47.994483 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1704b13-5bfa-4bbc-8ce6-eb27efa3700d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:47.995149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1704b13-5bfa-4bbc-8ce6-eb27efa3700d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:47.996960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1704b13-5bfa-4bbc-8ce6-eb27efa3700d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:47.996960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1704b13-5bfa-4bbc-8ce6-eb27efa3700d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:47.998891 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:48.018534 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1704b13-5bfa-4bbc-8ce6-eb27efa3700d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:48.020044 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1704b13-5bfa-4bbc-8ce6-eb27efa3700d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:48.034489 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-c1704b13-5bfa-4bbc-8ce6-eb27efa3700d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:48.034489 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:48.034489 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:48.034489 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:48.034489 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:48.034489 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:48.034489 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:48.034489 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:48.034489 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:48.034489 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:48.034489 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:48.034489 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:48.034489 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:30:48.034489 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:30:48.034489 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:30:48.035615 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:30:48.035615 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:48.035615 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:48.035615 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:48.035615 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:48.035615 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:48.035615 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:48.035615 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:48.035615 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:48.035615 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:48.035615 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:48.035615 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:48.035615 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:48.035615 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:48.035615 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:48.035615 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:48.035615 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:48.036789 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:30:48.036789 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:30:48.036789 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:30:48.036789 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:30:48.036789 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:30:48.036789 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:30:48.036789 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:48.036789 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:48.036789 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 403/804] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:30:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:48.037330 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1704b13-5bfa-4bbc-8ce6-eb27efa3700d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:48.072647 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:48.126341 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac3e52-cae9-44df-acad-ddd6b2cbd295 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:48.180653 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9274d-2e3d-4c90-93c3-f68c96b1b017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:48.181199 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1704b13-5bfa-4bbc-8ce6-eb27efa3700d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:48.184812 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9274d-2e3d-4c90-93c3-f68c96b1b017 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:48.184812 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9274d-2e3d-4c90-93c3-f68c96b1b017 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:48.184812 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9274d-2e3d-4c90-93c3-f68c96b1b017 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:48.190467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac9274d-2e3d-4c90-93c3-f68c96b1b017 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:48.191951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac9274d-2e3d-4c90-93c3-f68c96b1b017 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:48.208084 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-aac9274d-2e3d-4c90-93c3-f68c96b1b017 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 14:30:48.208084 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:48.208084 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:48.208084 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:48.208084 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:48.208084 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:48.208084 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:48.208084 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:48.208084 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:48.208084 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:48.208084 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:48.208084 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:48.208084 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:30:48.208084 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:30:48.208084 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:30:48.208084 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:30:48.209176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:48.209176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:48.209176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:30:48.209176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:48.209176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:30:48.209176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:48.209176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:48.209176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:48.209176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:48.209176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:48.209176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:48.209176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:48.209176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:48.209176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:48.209176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:48.209176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:48.209176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:48.209176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:48.210180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:48.210180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:48.210180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:30:48.210180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:30:48.210180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:30:48.210180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:30:48.210180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:30:48.210180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:30:48.210180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 14:30:48.210180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:48.215855 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 402/805] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:30:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:48.218142 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bccd0a-19c2-47f2-9890-1eca70d0b1f6 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:48.218439 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bccd0a-19c2-47f2-9890-1eca70d0b1f6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:48.218836 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bccd0a-19c2-47f2-9890-1eca70d0b1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:48.235797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f499334a-6eec-4c57-9514-1c14a9e5cb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:48.236707 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499334a-6eec-4c57-9514-1c14a9e5cb28 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:48.237159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499334a-6eec-4c57-9514-1c14a9e5cb28 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:48.237628 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499334a-6eec-4c57-9514-1c14a9e5cb28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:48.239105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-04bccd0a-19c2-47f2-9890-1eca70d0b1f6 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:48.240340 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-04bccd0a-19c2-47f2-9890-1eca70d0b1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:48.256210 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f499334a-6eec-4c57-9514-1c14a9e5cb28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:48.259310 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f499334a-6eec-4c57-9514-1c14a9e5cb28 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:48.284280 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-f499334a-6eec-4c57-9514-1c14a9e5cb28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:48.284280 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:48.284280 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:48.284280 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:48.284280 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:48.284280 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:48.284280 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:48.284280 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:48.284280 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:48.284280 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:48.284280 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:48.284280 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:48.284280 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:48.284280 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:48.284280 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:48.285213 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:48.285213 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:48.285213 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:48.285213 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:48.285213 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:48.285213 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:48.285213 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:48.285213 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:48.285213 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:48.285213 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:48.285213 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:48.285213 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:48.285213 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:48.285213 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:48.285213 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:48.285213 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:48.285213 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:48.286079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:48.286079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:48.286079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:48.286079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:48.286306 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 404/806] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:48.287464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f499334a-6eec-4c57-9514-1c14a9e5cb28 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:48.323435 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 403/807] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:30:48.324335 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bccd0a-19c2-47f2-9890-1eca70d0b1f6 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:48.345936 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:48.407766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:48.449492 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f499334a-6eec-4c57-9514-1c14a9e5cb28 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:48.471051 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bccd0a-19c2-47f2-9890-1eca70d0b1f6 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:48.502459 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2f4d6-af24-40b2-a40b-1503f5633908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:48.503288 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2f4d6-af24-40b2-a40b-1503f5633908 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:48.503615 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2f4d6-af24-40b2-a40b-1503f5633908 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:48.503895 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2f4d6-af24-40b2-a40b-1503f5633908 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:48.504548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a2f4d6-af24-40b2-a40b-1503f5633908 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:48.506104 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a2f4d6-af24-40b2-a40b-1503f5633908 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:48.519189 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-a7a2f4d6-af24-40b2-a40b-1503f5633908 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 14:30:48.519189 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:48.519189 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:30:48.519189 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:30:48.519189 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:30:48.519189 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:30:48.519189 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:30:48.519189 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:30:48.519189 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 14:30:48.519189 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:48.519189 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:30:48.519189 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:48.519189 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:48.519189 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:48.519189 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:48.519189 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:48.520231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:48.520231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:48.520231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:48.520231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:48.520231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:48.520231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:48.520231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:48.520231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:30:48.520231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:30:48.520231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:30:48.520231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:30:48.520231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:48.520231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:48.520231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:48.520231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:48.520231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:48.521098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:48.521098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:48.521098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:48.521098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:48.521098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:48.521098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:48.521098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:48.521098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:48.521098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:48.521098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:48.521098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:48.521098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:30:48.521098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:30:48.521098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 14:30:48.521098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:48.525851 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 405/808] 10.222.0.97 () {56 vars in 1129 bytes} [Tue Jul 26 14:30:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:48.526615 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2f4d6-af24-40b2-a40b-1503f5633908 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:48.530264 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208dda38-4c43-40e4-ac4c-bdffb4481a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:48.530542 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208dda38-4c43-40e4-ac4c-bdffb4481a5e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:48.530828 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208dda38-4c43-40e4-ac4c-bdffb4481a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:48.532014 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208dda38-4c43-40e4-ac4c-bdffb4481a5e None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:48.532014 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208dda38-4c43-40e4-ac4c-bdffb4481a5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:48.532412 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208dda38-4c43-40e4-ac4c-bdffb4481a5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:48.533619 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208dda38-4c43-40e4-ac4c-bdffb4481a5e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:48.548044 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 404/809] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:30:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:48.549349 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208dda38-4c43-40e4-ac4c-bdffb4481a5e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:48.622288 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:48.643927 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2f4d6-af24-40b2-a40b-1503f5633908 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:48.657304 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208dda38-4c43-40e4-ac4c-bdffb4481a5e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:48.683157 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2dcf0c-e293-4b08-9033-9c47fcd4bcf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:48.683808 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2dcf0c-e293-4b08-9033-9c47fcd4bcf7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:48.684126 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2dcf0c-e293-4b08-9033-9c47fcd4bcf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:48.684402 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2dcf0c-e293-4b08-9033-9c47fcd4bcf7 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:48.685193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2dcf0c-e293-4b08-9033-9c47fcd4bcf7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:48.685756 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2dcf0c-e293-4b08-9033-9c47fcd4bcf7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:48.686907 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2dcf0c-e293-4b08-9033-9c47fcd4bcf7 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:48.693404 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d4714-2c54-4eb2-bd81-1353989192d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:48.694033 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d4714-2c54-4eb2-bd81-1353989192d9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:48.694840 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d4714-2c54-4eb2-bd81-1353989192d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:48.695137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d4714-2c54-4eb2-bd81-1353989192d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:48.695861 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d4714-2c54-4eb2-bd81-1353989192d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:48.696178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d4714-2c54-4eb2-bd81-1353989192d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:48.696895 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:48.697785 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d4714-2c54-4eb2-bd81-1353989192d9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:48.706823 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 406/810] 10.222.0.97 () {56 vars in 1135 bytes} [Tue Jul 26 14:30:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:48.707952 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2dcf0c-e293-4b08-9033-9c47fcd4bcf7 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:48.715295 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 405/811] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:48.716413 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d4714-2c54-4eb2-bd81-1353989192d9 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:48.820618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2dcf0c-e293-4b08-9033-9c47fcd4bcf7 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:48.831870 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d4714-2c54-4eb2-bd81-1353989192d9 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:48.860085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:48.862212 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5153257d-ccc3-402e-983c-cc84cef11c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:48.862873 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153257d-ccc3-402e-983c-cc84cef11c5d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:48.863162 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153257d-ccc3-402e-983c-cc84cef11c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:48.863460 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153257d-ccc3-402e-983c-cc84cef11c5d None admin] PATH_INFO: `/v3/services/network` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:48.864179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5153257d-ccc3-402e-983c-cc84cef11c5d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:48.865558 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5153257d-ccc3-402e-983c-cc84cef11c5d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:48.870718 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8720bcc4-700a-4667-92db-0833da3393c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:48.874996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8720bcc4-700a-4667-92db-0833da3393c2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:48.874996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8720bcc4-700a-4667-92db-0833da3393c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:48.874996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8720bcc4-700a-4667-92db-0833da3393c2 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:48.874996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8720bcc4-700a-4667-92db-0833da3393c2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:48.874996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8720bcc4-700a-4667-92db-0833da3393c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:48.874996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8720bcc4-700a-4667-92db-0833da3393c2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:48.876171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-5153257d-ccc3-402e-983c-cc84cef11c5d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 14:30:48.876171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:48.876171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:30:48.876171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:30:48.876171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:30:48.876171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:30:48.876171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:30:48.876171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:30:48.876171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 14:30:48.876171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:48.876171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:30:48.876171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:48.876171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:48.876171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:48.876171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:48.876171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:48.876171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:48.877254 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:48.877254 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:48.877254 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:48.877254 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:48.877254 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:48.877254 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:48.877254 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:30:48.877254 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:30:48.877254 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:30:48.877254 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:30:48.877254 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:48.877254 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:48.877254 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:48.877254 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:48.877254 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:48.877254 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:48.878090 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:48.878090 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:48.878090 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:48.878090 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:48.878090 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:48.878090 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:48.878090 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:48.878090 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:48.878090 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:48.878090 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:48.878090 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:30:48.878090 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:30:48.878090 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 14:30:48.878090 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:48.878090 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 407/812] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:30:48 2022] GET /identity/v3/services/network => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:48.878867 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5153257d-ccc3-402e-983c-cc84cef11c5d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:48.884552 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 406/813] 10.222.0.97 () {56 vars in 1124 bytes} [Tue Jul 26 14:30:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:48.884552 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8720bcc4-700a-4667-92db-0833da3393c2 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:48.942746 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:48.988837 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5153257d-ccc3-402e-983c-cc84cef11c5d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:49.001349 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8720bcc4-700a-4667-92db-0833da3393c2 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:49.031526 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa31597-ae29-4ed2-a7bf-ae9eaf04e41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:49.032520 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa31597-ae29-4ed2-a7bf-ae9eaf04e41f None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:49.032965 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa31597-ae29-4ed2-a7bf-ae9eaf04e41f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:49.033242 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa31597-ae29-4ed2-a7bf-ae9eaf04e41f None admin] PATH_INFO: `/v3/projects/c4c8da3011294783b2180ade1bfd9366/users/5179c3c4eaaa41f3beb79ba7f1993478/roles/9074ca9226f8483c885f32fa868a1acc` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:49.042756 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2bb0f-5996-4b0a-8ad1-c7c682e2fc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:49.043434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2bb0f-5996-4b0a-8ad1-c7c682e2fc5a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:49.043719 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2bb0f-5996-4b0a-8ad1-c7c682e2fc5a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:49.043990 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2bb0f-5996-4b0a-8ad1-c7c682e2fc5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:49.049622 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa31597-ae29-4ed2-a7bf-ae9eaf04e41f None admin] RBAC: Authorizing `identity:create_grant(project_id=c4c8da3011294783b2180ade1bfd9366, user_id=5179c3c4eaaa41f3beb79ba7f1993478, role_id=9074ca9226f8483c885f32fa868a1acc)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:49.051283 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa31597-ae29-4ed2-a7bf-ae9eaf04e41f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:49.060867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2bb0f-5996-4b0a-8ad1-c7c682e2fc5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:49.062435 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2bb0f-5996-4b0a-8ad1-c7c682e2fc5a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:49.086058 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-52a2bb0f-5996-4b0a-8ad1-c7c682e2fc5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:49.086058 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:49.086058 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:49.086058 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:49.086058 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:49.086058 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:49.086058 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:49.086058 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:49.086058 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:49.086058 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:49.086058 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:49.086058 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:49.086058 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:49.086058 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:49.086058 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:49.087143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:49.087143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:49.087143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:49.087143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:49.087143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:49.087143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:49.087143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:49.087143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:49.087143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:49.087143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:49.087143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:49.087143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:49.087143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:49.087143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:49.087143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:49.087143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:49.087143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:49.088162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:49.088162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:49.088162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:49.088162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:49.088162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 407/814] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:49.088550 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2bb0f-5996-4b0a-8ad1-c7c682e2fc5a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:49.138246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:49.169221 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 408/815] 10.222.0.97 () {58 vars in 1439 bytes} [Tue Jul 26 14:30:48 2022] PUT /identity/v3/projects/c4c8da3011294783b2180ade1bfd9366/users/5179c3c4eaaa41f3beb79ba7f1993478/roles/9074ca9226f8483c885f32fa868a1acc => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:30:49.170564 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa31597-ae29-4ed2-a7bf-ae9eaf04e41f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:49.230619 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2bb0f-5996-4b0a-8ad1-c7c682e2fc5a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:49.290319 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851977d-d065-4bc3-bca0-e092580b67f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:49.291176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851977d-d065-4bc3-bca0-e092580b67f6 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:49.291690 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851977d-d065-4bc3-bca0-e092580b67f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:49.292151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851977d-d065-4bc3-bca0-e092580b67f6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:49.301431 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851977d-d065-4bc3-bca0-e092580b67f6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:49.308979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851977d-d065-4bc3-bca0-e092580b67f6 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:49.318158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-8851977d-d065-4bc3-bca0-e092580b67f6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:30:49.318158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:49.318158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:49.318158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:49.318158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:49.318158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:49.318158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:49.318158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:49.318158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:49.318158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:49.318158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:49.318158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:49.318158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:30:49.318158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:30:49.318158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:30:49.318158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:30:49.319814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:49.319814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:49.319814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:30:49.319814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:49.319814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:30:49.319814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:49.319814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:49.319814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:49.319814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:49.319814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:49.319814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:49.319814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:49.319814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:49.319814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:49.319814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:49.319814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:49.319814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:49.319814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:49.321641 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:49.321641 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:49.321641 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:30:49.321641 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:30:49.321641 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:30:49.321641 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:30:49.321641 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:30:49.321641 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:30:49.321641 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:30:49.321641 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:49.325067 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 408/816] 10.222.0.97 () {56 vars in 1093 bytes} [Tue Jul 26 14:30:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:49.326206 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851977d-d065-4bc3-bca0-e092580b67f6 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:49.367557 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa31597-ae29-4ed2-a7bf-ae9eaf04e41f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:49.385782 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:49.417005 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c5ab5-5211-4e4c-b475-a30834355561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:49.417914 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c5ab5-5211-4e4c-b475-a30834355561 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:49.418202 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c5ab5-5211-4e4c-b475-a30834355561 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:49.418481 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c5ab5-5211-4e4c-b475-a30834355561 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:49.419171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c5ab5-5211-4e4c-b475-a30834355561 None admin] RBAC: Adding query filter params (name=network) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:49.419515 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c5ab5-5211-4e4c-b475-a30834355561 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:49.420888 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c5ab5-5211-4e4c-b475-a30834355561 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:49.430869 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 409/817] 10.222.0.97 () {56 vars in 1112 bytes} [Tue Jul 26 14:30:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:49.431945 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c5ab5-5211-4e4c-b475-a30834355561 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:49.480384 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851977d-d065-4bc3-bca0-e092580b67f6 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:49.483509 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:49.528339 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fa04d-dca7-4190-9d13-aee5ab403d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:49.529901 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fa04d-dca7-4190-9d13-aee5ab403d84 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:49.530397 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fa04d-dca7-4190-9d13-aee5ab403d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:49.531111 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fa04d-dca7-4190-9d13-aee5ab403d84 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:49.532246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21fa04d-dca7-4190-9d13-aee5ab403d84 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:49.533060 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21fa04d-dca7-4190-9d13-aee5ab403d84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:49.535019 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21fa04d-dca7-4190-9d13-aee5ab403d84 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:49.548197 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 409/818] 10.222.0.97 () {56 vars in 1124 bytes} [Tue Jul 26 14:30:49 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:49.549110 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fa04d-dca7-4190-9d13-aee5ab403d84 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:49.589659 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c5ab5-5211-4e4c-b475-a30834355561 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:49.636730 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979af2e6-e15e-4065-97c4-e6c7610e0fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:49.637878 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979af2e6-e15e-4065-97c4-e6c7610e0fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:49.639158 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979af2e6-e15e-4065-97c4-e6c7610e0fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:49.639734 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979af2e6-e15e-4065-97c4-e6c7610e0fd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:49.640890 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979af2e6-e15e-4065-97c4-e6c7610e0fd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:49.641412 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979af2e6-e15e-4065-97c4-e6c7610e0fd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:49.643193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979af2e6-e15e-4065-97c4-e6c7610e0fd5 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:49.663697 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 410/819] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:49.665303 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979af2e6-e15e-4065-97c4-e6c7610e0fd5 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:49.683125 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fa04d-dca7-4190-9d13-aee5ab403d84 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:49.707852 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:49.730749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a578b-9f31-466d-88ed-4f37f07abd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:49.731446 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a578b-9f31-466d-88ed-4f37f07abd3a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:49.731742 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a578b-9f31-466d-88ed-4f37f07abd3a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:49.732027 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a578b-9f31-466d-88ed-4f37f07abd3a None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:49.732837 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a578b-9f31-466d-88ed-4f37f07abd3a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:49.733156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a578b-9f31-466d-88ed-4f37f07abd3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:49.734860 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a578b-9f31-466d-88ed-4f37f07abd3a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:49.765462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 410/820] 10.222.0.97 () {56 vars in 1133 bytes} [Tue Jul 26 14:30:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:49.766551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a578b-9f31-466d-88ed-4f37f07abd3a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:49.791112 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979af2e6-e15e-4065-97c4-e6c7610e0fd5 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:49.811248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:49.840078 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3641a4d-9988-4188-ac32-f9c9f5da3842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:49.842066 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3641a4d-9988-4188-ac32-f9c9f5da3842 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:49.843965 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3641a4d-9988-4188-ac32-f9c9f5da3842 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:49.844355 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3641a4d-9988-4188-ac32-f9c9f5da3842 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:49.845301 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3641a4d-9988-4188-ac32-f9c9f5da3842 None admin] RBAC: Adding query filter params (type=network) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:49.845698 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3641a4d-9988-4188-ac32-f9c9f5da3842 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:49.847067 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3641a4d-9988-4188-ac32-f9c9f5da3842 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:49.862595 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 411/821] 10.222.0.97 () {56 vars in 1112 bytes} [Tue Jul 26 14:30:49 2022] GET /identity/v3/services?type=network => generated 350 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:49.863862 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3641a4d-9988-4188-ac32-f9c9f5da3842 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:49.918177 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a578b-9f31-466d-88ed-4f37f07abd3a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:49.964125 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab609ef-a845-40ef-af95-3f75785b92e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:49.965295 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab609ef-a845-40ef-af95-3f75785b92e6 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:49.966178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab609ef-a845-40ef-af95-3f75785b92e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:49.966178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab609ef-a845-40ef-af95-3f75785b92e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:49.966905 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab609ef-a845-40ef-af95-3f75785b92e6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:49.968372 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab609ef-a845-40ef-af95-3f75785b92e6 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:49.983796 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:49.987886 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3641a4d-9988-4188-ac32-f9c9f5da3842 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:50.034318 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0fbe11-a4f6-4abb-8da5-acfa6113aabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:50.035048 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fbe11-a4f6-4abb-8da5-acfa6113aabe None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:50.035334 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fbe11-a4f6-4abb-8da5-acfa6113aabe None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:50.035719 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fbe11-a4f6-4abb-8da5-acfa6113aabe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:50.043239 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0fbe11-a4f6-4abb-8da5-acfa6113aabe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:50.044989 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0fbe11-a4f6-4abb-8da5-acfa6113aabe None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:50.058771 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-dc0fbe11-a4f6-4abb-8da5-acfa6113aabe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:50.058771 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:50.058771 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:50.058771 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:50.058771 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:50.058771 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:50.058771 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:50.058771 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:50.058771 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:50.058771 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:50.058771 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:50.058771 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:50.058771 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:30:50.058771 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:30:50.058771 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:30:50.060025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:30:50.060025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:50.060025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:50.060025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:50.060025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:50.060025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:50.060025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:50.060025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:50.060025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:50.060025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:50.060025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:50.060025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:50.060025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:50.060025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:50.060025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:50.060025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:50.060025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:50.061244 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:30:50.061244 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:30:50.061244 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:30:50.061244 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:30:50.061244 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:30:50.061244 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:30:50.061244 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:50.061244 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:50.061244 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 412/822] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:30:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:50.061936 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0fbe11-a4f6-4abb-8da5-acfa6113aabe None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:50.131285 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 411/823] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:30:49 2022] POST /identity/v3/endpoints => generated 346 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:30:50.132663 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab609ef-a845-40ef-af95-3f75785b92e6 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:50.248446 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0fbe11-a4f6-4abb-8da5-acfa6113aabe None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:50.328519 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:50.349454 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7cf3f-4d46-4749-9df4-10ac5a7601ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:50.350103 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7cf3f-4d46-4749-9df4-10ac5a7601ed None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:50.350468 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7cf3f-4d46-4749-9df4-10ac5a7601ed None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:50.350715 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7cf3f-4d46-4749-9df4-10ac5a7601ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:50.375100 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab609ef-a845-40ef-af95-3f75785b92e6 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:50.412515 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:50.456191 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd28c6-686f-4766-84f2-90a7968f39b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:50.458262 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd28c6-686f-4766-84f2-90a7968f39b8 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:50.458988 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd28c6-686f-4766-84f2-90a7968f39b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:50.459785 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd28c6-686f-4766-84f2-90a7968f39b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:50.468014 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd28c6-686f-4766-84f2-90a7968f39b8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:50.469896 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd28c6-686f-4766-84f2-90a7968f39b8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:50.518991 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7cf3f-4d46-4749-9df4-10ac5a7601ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:50.520665 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7cf3f-4d46-4749-9df4-10ac5a7601ed None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:50.577317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 412/824] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:30:50 2022] POST /identity/v3/endpoints => generated 327 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:30:50.580514 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd28c6-686f-4766-84f2-90a7968f39b8 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:50.761694 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd28c6-686f-4766-84f2-90a7968f39b8 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:50.804337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 413/825] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:30:50 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 743 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:30:50.810035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7cf3f-4d46-4749-9df4-10ac5a7601ed None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:50.815691 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:50.832202 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a0202-636f-4d0c-96db-617d06ddab8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:50.833449 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a0202-636f-4d0c-96db-617d06ddab8c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:50.833449 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a0202-636f-4d0c-96db-617d06ddab8c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:50.837863 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a0202-636f-4d0c-96db-617d06ddab8c None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:50.837863 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a0202-636f-4d0c-96db-617d06ddab8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:50.837863 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a0202-636f-4d0c-96db-617d06ddab8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:50.838185 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a0202-636f-4d0c-96db-617d06ddab8c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:50.856669 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 413/826] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:30:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:50.863494 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a0202-636f-4d0c-96db-617d06ddab8c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:50.890424 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:50.955826 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7cf3f-4d46-4749-9df4-10ac5a7601ed None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:51.003040 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fcc977-eb19-4732-973f-d2cd52bb439d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:51.004044 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fcc977-eb19-4732-973f-d2cd52bb439d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:51.004333 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fcc977-eb19-4732-973f-d2cd52bb439d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:51.004878 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fcc977-eb19-4732-973f-d2cd52bb439d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:51.017355 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fcc977-eb19-4732-973f-d2cd52bb439d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:51.020000 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fcc977-eb19-4732-973f-d2cd52bb439d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:51.040114 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a0202-636f-4d0c-96db-617d06ddab8c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:51.040306 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-66fcc977-eb19-4732-973f-d2cd52bb439d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:51.040306 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:51.040306 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:51.040306 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:51.040306 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:51.040306 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:51.040306 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:51.040306 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:51.040306 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:51.040306 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:51.040306 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:51.040306 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:51.040306 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:51.040306 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:51.040306 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:51.041111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:51.041111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:51.041111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:51.041111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:51.041111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:51.041111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:51.041111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:51.041111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:51.041111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:51.041111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:51.041111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:51.041111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:51.041111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:51.041111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:51.041111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:51.041111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:51.041111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:51.041970 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:51.041970 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:51.041970 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:51.041970 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:51.049806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 414/827] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:51.053468 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fcc977-eb19-4732-973f-d2cd52bb439d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:51.086735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60b5b7-751a-48a6-a58a-69c56ae64238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:51.086735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60b5b7-751a-48a6-a58a-69c56ae64238 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:51.086735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60b5b7-751a-48a6-a58a-69c56ae64238 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:51.086735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60b5b7-751a-48a6-a58a-69c56ae64238 None admin] PATH_INFO: `/v3/projects/c4c8da3011294783b2180ade1bfd9366/users/85122c4052fd4ba19a2dae185a707caf/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:51.090131 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60b5b7-751a-48a6-a58a-69c56ae64238 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4c8da3011294783b2180ade1bfd9366, user_id=85122c4052fd4ba19a2dae185a707caf, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:51.091637 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60b5b7-751a-48a6-a58a-69c56ae64238 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:51.215447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fcc977-eb19-4732-973f-d2cd52bb439d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:51.223863 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:51.259434 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288f842-352e-41d9-ad8b-f04c8680b730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:51.259434 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288f842-352e-41d9-ad8b-f04c8680b730 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:51.259434 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288f842-352e-41d9-ad8b-f04c8680b730 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:51.259434 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288f842-352e-41d9-ad8b-f04c8680b730 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:51.260087 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288f842-352e-41d9-ad8b-f04c8680b730 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:51.260219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288f842-352e-41d9-ad8b-f04c8680b730 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:51.263437 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288f842-352e-41d9-ad8b-f04c8680b730 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:51.282241 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 415/828] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:51.283269 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288f842-352e-41d9-ad8b-f04c8680b730 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:51.399538 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 414/829] 10.222.0.97 () {58 vars in 1439 bytes} [Tue Jul 26 14:30:50 2022] PUT /identity/v3/projects/c4c8da3011294783b2180ade1bfd9366/users/85122c4052fd4ba19a2dae185a707caf/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:30:51.405406 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288f842-352e-41d9-ad8b-f04c8680b730 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:51.451036 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d4fed-d015-4fb9-ab6d-5de1d3bebfad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:51.451036 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d4fed-d015-4fb9-ab6d-5de1d3bebfad None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:51.451036 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d4fed-d015-4fb9-ab6d-5de1d3bebfad None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:51.451036 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d4fed-d015-4fb9-ab6d-5de1d3bebfad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:51.458088 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d4fed-d015-4fb9-ab6d-5de1d3bebfad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:51.459704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d4fed-d015-4fb9-ab6d-5de1d3bebfad None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:51.476716 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-6e0d4fed-d015-4fb9-ab6d-5de1d3bebfad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:51.476716 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:51.476716 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:51.476716 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:51.476716 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:51.476716 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:51.476716 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:51.476716 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:51.476716 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:51.476716 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:51.476716 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:51.476716 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:51.476716 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:30:51.476716 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:30:51.476716 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:30:51.480251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:30:51.480251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:51.480251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:51.480251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:51.480251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:51.480251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:51.480251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:51.480251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:51.480251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:51.480251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:51.480251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:51.480251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:51.480251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:51.480251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:51.480251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:51.480251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:51.480251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:51.483706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:30:51.483706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:30:51.483706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:30:51.483706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:30:51.483706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:30:51.483706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:30:51.483706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:51.483706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:51.483706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 416/830] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:30:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:51.492469 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60b5b7-751a-48a6-a58a-69c56ae64238 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:51.564043 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:51.622676 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60b5b7-751a-48a6-a58a-69c56ae64238 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:51.667645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9f1d2-61d7-4e40-b6cb-b2cef77625be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:51.668793 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9f1d2-61d7-4e40-b6cb-b2cef77625be None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:51.669245 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9f1d2-61d7-4e40-b6cb-b2cef77625be None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:51.669681 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9f1d2-61d7-4e40-b6cb-b2cef77625be None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:51.670608 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9f1d2-61d7-4e40-b6cb-b2cef77625be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:51.671085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9f1d2-61d7-4e40-b6cb-b2cef77625be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:51.672994 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9f1d2-61d7-4e40-b6cb-b2cef77625be None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:51.690404 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 415/831] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:30:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:51.697525 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d4fed-d015-4fb9-ab6d-5de1d3bebfad None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:51.731785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:51.816919 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d4fed-d015-4fb9-ab6d-5de1d3bebfad None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:51.859345 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1bfbf-9dda-4279-ae90-2dbc9ed6ccff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:51.860472 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1bfbf-9dda-4279-ae90-2dbc9ed6ccff None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:51.860926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1bfbf-9dda-4279-ae90-2dbc9ed6ccff None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:51.861384 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1bfbf-9dda-4279-ae90-2dbc9ed6ccff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:51.862351 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1bfbf-9dda-4279-ae90-2dbc9ed6ccff None admin] RBAC: Adding query filter params (role.id=51b2d10dfffb4b75b186f3d8b15e2a78, scope.project.id=c4c8da3011294783b2180ade1bfd9366, user.id=266abef6cda5495b897170f92043aa03) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:51.862844 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1bfbf-9dda-4279-ae90-2dbc9ed6ccff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:51.864680 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1bfbf-9dda-4279-ae90-2dbc9ed6ccff None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:51.876091 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 417/832] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:30:51 2022] GET /identity/v3/role_assignments?user.id=266abef6cda5495b897170f92043aa03&scope.project.id=c4c8da3011294783b2180ade1bfd9366&role.id=51b2d10dfffb4b75b186f3d8b15e2a78 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:30:51.974297 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151be7f6-30b1-4fe3-84ad-258b43022547 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:51.974297 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151be7f6-30b1-4fe3-84ad-258b43022547 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:51.974297 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151be7f6-30b1-4fe3-84ad-258b43022547 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:51.979454 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 416/833] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:51.987847 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967adee8-6514-4f0b-bd7a-890e252d64b9 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:51.988279 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967adee8-6514-4f0b-bd7a-890e252d64b9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:51.988451 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967adee8-6514-4f0b-bd7a-890e252d64b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:52.011932 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-967adee8-6514-4f0b-bd7a-890e252d64b9 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:52.013176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-967adee8-6514-4f0b-bd7a-890e252d64b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:52.115293 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 418/834] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:51 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:30:52.157621 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151be7f6-30b1-4fe3-84ad-258b43022547 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:52.326212 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151be7f6-30b1-4fe3-84ad-258b43022547 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:52.364975 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:52.400334 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140d90b-cb52-475e-930e-29df1761bee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:52.401437 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140d90b-cb52-475e-930e-29df1761bee9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:52.401922 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140d90b-cb52-475e-930e-29df1761bee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:52.402348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140d90b-cb52-475e-930e-29df1761bee9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:52.445710 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140d90b-cb52-475e-930e-29df1761bee9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:52.454249 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-f140d90b-cb52-475e-930e-29df1761bee9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:52.454249 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:52.454249 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:52.454249 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:52.454249 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:52.454249 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:52.454249 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:52.454249 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:52.454249 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:52.454249 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:52.454249 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:52.454249 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:52.454249 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:52.454249 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:52.454249 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:52.454249 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:52.455246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:52.455246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:52.455246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:52.455246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:52.455246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:52.455246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:52.455246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:52.455246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:52.455246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:52.455246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:52.455246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:52.455246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:52.455246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:52.455246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:52.455246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:52.455246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:52.456139 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:52.456139 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:52.456139 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:52.456139 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:52.456139 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:52.456139 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:52.456139 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:52.456139 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:52.456139 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:52.456139 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:52.456139 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:30:52.456139 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:52.456790 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 417/835] 10.222.0.97 () {56 vars in 1102 bytes} [Tue Jul 26 14:30:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:52.465047 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967adee8-6514-4f0b-bd7a-890e252d64b9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:52.516347 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:52.662418 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967adee8-6514-4f0b-bd7a-890e252d64b9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:52.722624 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec52988-ebc4-40a8-940f-cd7987b5b7a1 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:52.723098 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec52988-ebc4-40a8-940f-cd7987b5b7a1 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:52.723256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec52988-ebc4-40a8-940f-cd7987b5b7a1 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:52.725257 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 418/836] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:30:52.726431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc87aecc-7088-476e-9529-26142b2dc90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:52.727469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc87aecc-7088-476e-9529-26142b2dc90d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:52.727957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc87aecc-7088-476e-9529-26142b2dc90d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:52.728450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc87aecc-7088-476e-9529-26142b2dc90d None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:52.729768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc87aecc-7088-476e-9529-26142b2dc90d None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:52.730429 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc87aecc-7088-476e-9529-26142b2dc90d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:52.732171 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19969986-8b3e-4d65-84a1-fdd7eff35f9f None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:52.732606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc87aecc-7088-476e-9529-26142b2dc90d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:52.732935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19969986-8b3e-4d65-84a1-fdd7eff35f9f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:52.733626 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19969986-8b3e-4d65-84a1-fdd7eff35f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:52.767466 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 419/837] 10.222.0.97 () {56 vars in 1103 bytes} [Tue Jul 26 14:30:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:52.776087 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc87aecc-7088-476e-9529-26142b2dc90d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:52.783385 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-19969986-8b3e-4d65-84a1-fdd7eff35f9f None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:52.785225 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-19969986-8b3e-4d65-84a1-fdd7eff35f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:52.904882 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:52.927820 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc87aecc-7088-476e-9529-26142b2dc90d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:52.935161 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 419/838] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:52 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:30:52.974603 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19969986-8b3e-4d65-84a1-fdd7eff35f9f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:52.984902 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae585cbb-8ddc-4ca4-9672-090a8a9d0831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:52.985935 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae585cbb-8ddc-4ca4-9672-090a8a9d0831 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:52.986404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae585cbb-8ddc-4ca4-9672-090a8a9d0831 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:52.986877 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae585cbb-8ddc-4ca4-9672-090a8a9d0831 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:53.012653 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae585cbb-8ddc-4ca4-9672-090a8a9d0831 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:53.015682 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae585cbb-8ddc-4ca4-9672-090a8a9d0831 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:53.051441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-ae585cbb-8ddc-4ca4-9672-090a8a9d0831 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:53.051441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:53.051441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:53.051441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:53.051441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:53.051441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:53.051441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:53.051441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:53.051441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:53.051441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:53.051441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:53.051441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:53.051441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:53.051441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:53.051441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:53.053373 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:53.053373 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:53.053373 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:53.053373 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:53.053373 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:53.053373 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:53.053373 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:53.053373 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:53.053373 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:53.053373 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:53.053373 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:53.053373 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:53.053373 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:53.053373 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:53.053373 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:53.053373 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:53.053373 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:53.055624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:53.055624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:53.055624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:53.055624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:53.055624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 420/839] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:53.063140 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae585cbb-8ddc-4ca4-9672-090a8a9d0831 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:53.259562 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19969986-8b3e-4d65-84a1-fdd7eff35f9f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:53.266923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:53.390561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae585cbb-8ddc-4ca4-9672-090a8a9d0831 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:53.401616 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b70ca2-c545-4fc7-963f-b907905f9f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:53.402709 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b70ca2-c545-4fc7-963f-b907905f9f4e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:53.403199 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b70ca2-c545-4fc7-963f-b907905f9f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:53.403708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b70ca2-c545-4fc7-963f-b907905f9f4e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:53.404810 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b70ca2-c545-4fc7-963f-b907905f9f4e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:53.407311 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b70ca2-c545-4fc7-963f-b907905f9f4e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:53.445395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-25b70ca2-c545-4fc7-963f-b907905f9f4e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 14:30:53.445395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:53.445395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:30:53.445395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:30:53.445395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:30:53.445395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:30:53.445395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:30:53.445395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:30:53.445395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 14:30:53.445395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:53.445395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:30:53.445395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:53.445395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:53.445395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:53.445395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:53.445395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:53.445395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:53.446419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:53.446419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:53.446419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:53.446419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:53.446419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:53.446419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:53.446419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:30:53.446419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:30:53.446419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:30:53.446419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:30:53.446419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:53.446419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:53.446419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:53.446419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:53.446419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:53.446419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:53.447220 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:53.447220 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:53.447220 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:53.447220 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:53.447220 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:53.447220 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:53.447220 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:53.447220 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:53.447220 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:53.447220 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:53.447220 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:30:53.447220 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:30:53.447220 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 14:30:53.447220 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:53.452282 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 420/840] 10.222.0.97 () {56 vars in 1114 bytes} [Tue Jul 26 14:30:52 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 478 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:53.454600 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b70ca2-c545-4fc7-963f-b907905f9f4e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:53.530422 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80895051-5d13-46df-a9dd-dcc152d5051c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:53.531478 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80895051-5d13-46df-a9dd-dcc152d5051c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:53.532430 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80895051-5d13-46df-a9dd-dcc152d5051c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:53.534524 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80895051-5d13-46df-a9dd-dcc152d5051c None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:53.536455 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80895051-5d13-46df-a9dd-dcc152d5051c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:53.537057 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80895051-5d13-46df-a9dd-dcc152d5051c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:53.538877 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80895051-5d13-46df-a9dd-dcc152d5051c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:53.543056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:53.578130 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 421/841] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 516 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:53.583246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80895051-5d13-46df-a9dd-dcc152d5051c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:53.768078 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b70ca2-c545-4fc7-963f-b907905f9f4e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:53.830605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80895051-5d13-46df-a9dd-dcc152d5051c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:53.894082 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f224c-4091-40aa-acc7-c3c08dca0c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:53.901327 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f224c-4091-40aa-acc7-c3c08dca0c3f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:53.901898 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f224c-4091-40aa-acc7-c3c08dca0c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:53.902369 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f224c-4091-40aa-acc7-c3c08dca0c3f None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:53.903599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f224c-4091-40aa-acc7-c3c08dca0c3f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:53.904111 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f224c-4091-40aa-acc7-c3c08dca0c3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:53.914871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f224c-4091-40aa-acc7-c3c08dca0c3f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:53.920558 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a14ab1-6cb4-4d3a-b3f7-95f988398205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:53.921599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a14ab1-6cb4-4d3a-b3f7-95f988398205 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:53.922065 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a14ab1-6cb4-4d3a-b3f7-95f988398205 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:53.922532 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a14ab1-6cb4-4d3a-b3f7-95f988398205 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:53.930870 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 421/842] 10.222.0.97 () {56 vars in 1114 bytes} [Tue Jul 26 14:30:53 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 477 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:53.937586 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f224c-4091-40aa-acc7-c3c08dca0c3f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:53.952464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a14ab1-6cb4-4d3a-b3f7-95f988398205 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:53.955025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a14ab1-6cb4-4d3a-b3f7-95f988398205 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:53.998978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-28a14ab1-6cb4-4d3a-b3f7-95f988398205 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 14:30:53.998978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:53.998978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:53.998978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:53.998978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:53.998978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:53.998978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:53.998978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:53.998978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:53.998978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:53.998978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:53.998978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:53.998978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:30:53.998978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:30:53.998978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:30:53.998978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:30:54.000989 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:54.000989 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:54.000989 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:30:54.000989 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:54.000989 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:30:54.000989 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:54.000989 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:54.000989 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:54.000989 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:54.000989 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:54.000989 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:54.000989 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:54.000989 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:54.000989 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:54.000989 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:54.000989 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:54.000989 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:54.000989 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:54.002940 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:54.002940 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:54.002940 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:30:54.002940 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:30:54.002940 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:30:54.002940 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:30:54.002940 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:30:54.002940 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:30:54.002940 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 14:30:54.002940 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:54.004678 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 422/843] 10.222.0.97 () {56 vars in 1117 bytes} [Tue Jul 26 14:30:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 421 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:54.008012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff39ef9-89f9-433c-9c96-8d5729bc8f79 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:54.008572 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff39ef9-89f9-433c-9c96-8d5729bc8f79 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:54.009452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:54.010907 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff39ef9-89f9-433c-9c96-8d5729bc8f79 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:54.013402 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 423/844] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:54 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:54.015054 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff39ef9-89f9-433c-9c96-8d5729bc8f79 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:54.109297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:54.156326 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f224c-4091-40aa-acc7-c3c08dca0c3f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:54.175247 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff39ef9-89f9-433c-9c96-8d5729bc8f79 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:54.223272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8bb32-bb17-4c44-91a7-39d89efe15e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:54.224859 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8bb32-bb17-4c44-91a7-39d89efe15e4 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:54.225914 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8bb32-bb17-4c44-91a7-39d89efe15e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:54.226700 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8bb32-bb17-4c44-91a7-39d89efe15e4 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:54.228205 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8bb32-bb17-4c44-91a7-39d89efe15e4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:54.229036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8bb32-bb17-4c44-91a7-39d89efe15e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:54.231685 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8bb32-bb17-4c44-91a7-39d89efe15e4 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:54.234462 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b59d25-dbce-47e1-806f-241181a8e43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:54.235099 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b59d25-dbce-47e1-806f-241181a8e43b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:54.235386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b59d25-dbce-47e1-806f-241181a8e43b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:54.235754 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b59d25-dbce-47e1-806f-241181a8e43b None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:54.236582 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b59d25-dbce-47e1-806f-241181a8e43b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:54.236908 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b59d25-dbce-47e1-806f-241181a8e43b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:54.238686 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b59d25-dbce-47e1-806f-241181a8e43b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:54.244310 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 422/845] 10.222.0.97 () {56 vars in 1114 bytes} [Tue Jul 26 14:30:53 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:30:54.247046 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2159f-5e74-4fad-b615-95d57d279ed3 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:54.248028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2159f-5e74-4fad-b615-95d57d279ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:54.248805 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2159f-5e74-4fad-b615-95d57d279ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:54.288979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-5ed2159f-5e74-4fad-b615-95d57d279ed3 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:54.291493 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5ed2159f-5e74-4fad-b615-95d57d279ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:54.294394 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 424/846] 10.222.0.97 () {56 vars in 1149 bytes} [Tue Jul 26 14:30:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:54.305399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b59d25-dbce-47e1-806f-241181a8e43b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:54.531434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 423/847] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:30:54.581776 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b59d25-dbce-47e1-806f-241181a8e43b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:54.585981 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2159f-5e74-4fad-b615-95d57d279ed3 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:54.598039 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:54.621063 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17ab09-77cd-4952-8e5e-a4b260a03de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:54.621986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17ab09-77cd-4952-8e5e-a4b260a03de7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:54.627003 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17ab09-77cd-4952-8e5e-a4b260a03de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:54.627003 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17ab09-77cd-4952-8e5e-a4b260a03de7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:54.638340 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:54.703440 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e17ab09-77cd-4952-8e5e-a4b260a03de7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:54.708347 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e17ab09-77cd-4952-8e5e-a4b260a03de7 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:54.784730 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2159f-5e74-4fad-b615-95d57d279ed3 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:54.843020 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f09bc0-9494-4dba-a996-7d9076740e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:54.843711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f09bc0-9494-4dba-a996-7d9076740e4f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:54.843996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f09bc0-9494-4dba-a996-7d9076740e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:54.844275 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f09bc0-9494-4dba-a996-7d9076740e4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:54.849449 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 425/848] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:30:54 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:30:54.851725 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3334ac-c5bc-45d6-9d51-cc536a97984d None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:54.852023 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3334ac-c5bc-45d6-9d51-cc536a97984d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:54.852343 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3334ac-c5bc-45d6-9d51-cc536a97984d None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:54.854173 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 426/849] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:54.854981 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3334ac-c5bc-45d6-9d51-cc536a97984d None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:54.862569 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f09bc0-9494-4dba-a996-7d9076740e4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:54.869248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-17f09bc0-9494-4dba-a996-7d9076740e4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:30:54.869248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:54.869248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:54.869248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:54.869248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:54.869248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:54.869248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:54.869248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:54.869248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:54.869248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:54.869248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:54.869248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:54.869248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:54.869248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:54.869248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:54.869248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:54.870178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:54.870178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:54.870178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:54.870178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:54.870178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:54.870178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:54.870178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:54.870178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:54.870178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:54.870178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:54.870178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:54.870178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:54.870178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:54.870178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:54.870178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:54.870178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:54.871619 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:54.871619 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:54.871619 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:54.871619 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:54.871619 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:54.871619 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:54.871619 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:54.871619 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:54.871619 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:54.871619 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:54.871619 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:30:54.871619 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:30:54.871619 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 424/850] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:30:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:30:54.872334 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede5a64-f8d4-4a35-920e-cce8a90cae5c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:54.872413 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede5a64-f8d4-4a35-920e-cce8a90cae5c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:54.872669 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede5a64-f8d4-4a35-920e-cce8a90cae5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:54.880478 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:54.897411 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-eede5a64-f8d4-4a35-920e-cce8a90cae5c None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:54.898643 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-eede5a64-f8d4-4a35-920e-cce8a90cae5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:54.988429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 425/851] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:30:54.988429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede5a64-f8d4-4a35-920e-cce8a90cae5c None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:55.005209 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:55.016325 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3334ac-c5bc-45d6-9d51-cc536a97984d None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:55.065985 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b15a99-3142-44e2-86f7-46b9ee597e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:55.067717 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b15a99-3142-44e2-86f7-46b9ee597e8d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:55.067717 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b15a99-3142-44e2-86f7-46b9ee597e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:55.067717 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b15a99-3142-44e2-86f7-46b9ee597e8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:55.082064 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b15a99-3142-44e2-86f7-46b9ee597e8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:55.084982 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b15a99-3142-44e2-86f7-46b9ee597e8d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:55.105640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-e2b15a99-3142-44e2-86f7-46b9ee597e8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:55.105640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:55.105640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:55.105640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:55.105640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:55.105640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:55.105640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:55.105640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:55.105640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:55.105640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:55.105640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:55.105640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:55.105640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:55.105640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:55.105640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:55.106780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:55.106780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:55.106780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:55.106780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:55.106780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:55.106780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:55.106780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:55.106780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:55.106780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:55.106780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:55.106780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:55.106780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:55.106780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:55.106780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:55.106780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:55.106780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:55.106780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:55.108016 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:55.108016 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:55.108016 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:55.108016 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:55.111586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 427/852] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:55.112485 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b15a99-3142-44e2-86f7-46b9ee597e8d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:55.127571 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede5a64-f8d4-4a35-920e-cce8a90cae5c None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:55.170346 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ab9fe-d017-4042-bcdf-63de536cb8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:55.171022 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ab9fe-d017-4042-bcdf-63de536cb8a7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:55.171261 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ab9fe-d017-4042-bcdf-63de536cb8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:55.171561 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ab9fe-d017-4042-bcdf-63de536cb8a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:55.187541 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ab9fe-d017-4042-bcdf-63de536cb8a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:55.187541 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ab9fe-d017-4042-bcdf-63de536cb8a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:55.187541 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ab9fe-d017-4042-bcdf-63de536cb8a7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:55.187541 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 426/853] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:30:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:55.187541 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ab9fe-d017-4042-bcdf-63de536cb8a7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:55.191664 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:55.235530 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b15a99-3142-44e2-86f7-46b9ee597e8d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:55.282762 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9946806e-3497-4075-bb9b-602063db54fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:55.284507 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946806e-3497-4075-bb9b-602063db54fb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:55.284507 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946806e-3497-4075-bb9b-602063db54fb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:55.284702 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946806e-3497-4075-bb9b-602063db54fb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:55.285546 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9946806e-3497-4075-bb9b-602063db54fb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:55.287447 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:55.287587 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9946806e-3497-4075-bb9b-602063db54fb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:55.297785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-9946806e-3497-4075-bb9b-602063db54fb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 14:30:55.297785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:55.297785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:30:55.297785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:30:55.297785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:30:55.297785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:30:55.297785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:30:55.297785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:30:55.297785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 14:30:55.297785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:55.297785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:30:55.297785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:55.297785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:55.297785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:55.297785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:55.297785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:55.298737 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:55.298737 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:55.298737 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:55.298737 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:55.298737 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:55.298737 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:55.298737 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:55.298737 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:30:55.298737 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:30:55.298737 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:30:55.298737 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:30:55.298737 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:55.298737 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:55.298737 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:55.298737 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:55.298737 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:55.299612 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:55.299612 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:55.299612 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:55.299612 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:55.299612 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:55.299612 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:55.299612 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:55.299612 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:55.299612 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:55.299612 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:55.299612 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:55.299612 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:30:55.299612 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:30:55.299612 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 14:30:55.299612 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:55.304895 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 428/854] 10.222.0.97 () {56 vars in 1132 bytes} [Tue Jul 26 14:30:55 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:55.305802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9946806e-3497-4075-bb9b-602063db54fb None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:55.321378 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ab9fe-d017-4042-bcdf-63de536cb8a7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:55.366133 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd105e-1dfa-4c73-99af-10c403e13186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:55.366774 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd105e-1dfa-4c73-99af-10c403e13186 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:55.367062 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd105e-1dfa-4c73-99af-10c403e13186 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:55.367339 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd105e-1dfa-4c73-99af-10c403e13186 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:55.368200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcd105e-1dfa-4c73-99af-10c403e13186 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:55.368509 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcd105e-1dfa-4c73-99af-10c403e13186 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:55.370480 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcd105e-1dfa-4c73-99af-10c403e13186 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:55.386596 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 427/855] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:55.387315 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd105e-1dfa-4c73-99af-10c403e13186 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:55.464108 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9946806e-3497-4075-bb9b-602063db54fb None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:55.497496 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:55.508282 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09e670-7dc2-44d1-b303-04c9e133d710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:55.509125 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09e670-7dc2-44d1-b303-04c9e133d710 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:55.509596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09e670-7dc2-44d1-b303-04c9e133d710 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:55.509880 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09e670-7dc2-44d1-b303-04c9e133d710 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:55.524769 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09e670-7dc2-44d1-b303-04c9e133d710 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:55.531233 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09e670-7dc2-44d1-b303-04c9e133d710 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:55.534414 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd105e-1dfa-4c73-99af-10c403e13186 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:55.542410 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-1f09e670-7dc2-44d1-b303-04c9e133d710 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:55.542410 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:55.542410 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:55.542410 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:55.542410 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:55.542410 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:55.542410 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:55.542410 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:55.542410 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:55.542410 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:55.542410 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:55.542410 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:55.542410 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:55.542410 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:55.542410 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:55.543685 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:55.543685 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:55.543685 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:55.543685 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:55.543685 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:55.543685 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:55.543685 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:55.543685 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:55.543685 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:55.543685 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:55.543685 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:55.543685 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:55.543685 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:55.543685 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:55.543685 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:55.543685 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:55.543685 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:55.544554 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:55.544554 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:55.544554 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:55.544554 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:55.547374 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 429/856] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:55.547374 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09e670-7dc2-44d1-b303-04c9e133d710 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:55.582347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1912172-bfc1-4ff6-bc84-69a7f0851745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:55.583011 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1912172-bfc1-4ff6-bc84-69a7f0851745 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:55.583305 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1912172-bfc1-4ff6-bc84-69a7f0851745 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:55.583621 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1912172-bfc1-4ff6-bc84-69a7f0851745 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:55.585086 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1912172-bfc1-4ff6-bc84-69a7f0851745 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:55.585680 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1912172-bfc1-4ff6-bc84-69a7f0851745 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:55.588633 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1912172-bfc1-4ff6-bc84-69a7f0851745 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:55.591945 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:55.601182 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 428/857] 10.222.0.97 () {56 vars in 1126 bytes} [Tue Jul 26 14:30:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:55.601182 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1912172-bfc1-4ff6-bc84-69a7f0851745 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:55.685559 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09e670-7dc2-44d1-b303-04c9e133d710 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:55.730168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46b8d5-eeaf-4d10-8e3f-56fe1d06ade9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:55.731061 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46b8d5-eeaf-4d10-8e3f-56fe1d06ade9 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:55.731565 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46b8d5-eeaf-4d10-8e3f-56fe1d06ade9 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:55.731960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46b8d5-eeaf-4d10-8e3f-56fe1d06ade9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:55.738021 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1912172-bfc1-4ff6-bc84-69a7f0851745 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:55.740132 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e46b8d5-eeaf-4d10-8e3f-56fe1d06ade9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:55.742371 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e46b8d5-eeaf-4d10-8e3f-56fe1d06ade9 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:55.754797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-5e46b8d5-eeaf-4d10-8e3f-56fe1d06ade9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:55.754797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:55.754797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:55.754797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:55.754797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:55.754797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:55.754797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:55.754797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:55.754797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:55.754797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:55.754797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:55.754797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:55.754797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:30:55.754797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:30:55.754797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:30:55.755778 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:30:55.755778 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:55.755778 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:55.755778 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:55.755778 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:55.755778 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:55.755778 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:55.755778 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:55.755778 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:55.755778 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:55.755778 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:55.755778 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:55.755778 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:55.755778 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:55.755778 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:55.755778 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:55.755778 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:55.757413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:30:55.757413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:30:55.757413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:30:55.757413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:30:55.757413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:30:55.757413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:30:55.757413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:55.757413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:55.757927 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 430/858] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:30:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:55.758597 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46b8d5-eeaf-4d10-8e3f-56fe1d06ade9 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:55.793094 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0df6f7-f743-424b-ac17-503cb1923f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:55.793737 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0df6f7-f743-424b-ac17-503cb1923f4b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:55.794182 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0df6f7-f743-424b-ac17-503cb1923f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:55.794472 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0df6f7-f743-424b-ac17-503cb1923f4b None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:55.795795 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0df6f7-f743-424b-ac17-503cb1923f4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:55.796246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0df6f7-f743-424b-ac17-503cb1923f4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:55.798078 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0df6f7-f743-424b-ac17-503cb1923f4b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:55.813210 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:55.815078 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 429/859] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:55.816030 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0df6f7-f743-424b-ac17-503cb1923f4b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:55.902988 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46b8d5-eeaf-4d10-8e3f-56fe1d06ade9 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:55.904548 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:55.951970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0df6f7-f743-424b-ac17-503cb1923f4b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:55.953083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c669d-8816-4c7e-b901-7ca26ec2d406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:55.953719 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c669d-8816-4c7e-b901-7ca26ec2d406 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:55.954959 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c669d-8816-4c7e-b901-7ca26ec2d406 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:55.955248 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c669d-8816-4c7e-b901-7ca26ec2d406 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:55.959644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8c669d-8816-4c7e-b901-7ca26ec2d406 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:55.959965 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8c669d-8816-4c7e-b901-7ca26ec2d406 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:55.961389 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8c669d-8816-4c7e-b901-7ca26ec2d406 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:55.975459 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 431/860] 10.222.0.97 () {56 vars in 1126 bytes} [Tue Jul 26 14:30:55 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:55.975459 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c669d-8816-4c7e-b901-7ca26ec2d406 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:56.013976 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c344744-bb7d-4780-a937-8e2e12aa6356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:56.014886 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c344744-bb7d-4780-a937-8e2e12aa6356 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:56.015809 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c344744-bb7d-4780-a937-8e2e12aa6356 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:56.016114 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c344744-bb7d-4780-a937-8e2e12aa6356 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:56.017061 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c344744-bb7d-4780-a937-8e2e12aa6356 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:56.017488 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c344744-bb7d-4780-a937-8e2e12aa6356 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:56.019265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c344744-bb7d-4780-a937-8e2e12aa6356 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:56.039141 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 430/861] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:30:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:56.050620 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c344744-bb7d-4780-a937-8e2e12aa6356 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:56.195895 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c669d-8816-4c7e-b901-7ca26ec2d406 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:56.200924 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:56.250357 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db118dfe-5c43-416c-b1f3-ad3f2a3ef2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:56.251370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db118dfe-5c43-416c-b1f3-ad3f2a3ef2a3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:56.251863 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db118dfe-5c43-416c-b1f3-ad3f2a3ef2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:56.252297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db118dfe-5c43-416c-b1f3-ad3f2a3ef2a3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:56.253607 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c344744-bb7d-4780-a937-8e2e12aa6356 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:56.266164 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db118dfe-5c43-416c-b1f3-ad3f2a3ef2a3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:56.269003 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db118dfe-5c43-416c-b1f3-ad3f2a3ef2a3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:56.278853 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:56.286286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-db118dfe-5c43-416c-b1f3-ad3f2a3ef2a3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 14:30:56.286286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:56.286286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:56.286286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:56.286286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:56.286286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:56.286286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:56.286286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:56.286286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:56.286286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:56.286286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:56.286286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:56.286286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:30:56.286286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:30:56.286286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:30:56.286286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:30:56.288240 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:56.288240 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:56.288240 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:30:56.288240 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:56.288240 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:30:56.288240 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:30:56.288240 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:56.288240 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:56.288240 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:56.288240 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:56.288240 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:56.288240 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:56.288240 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:56.288240 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:56.288240 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:56.288240 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:56.288240 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:56.288240 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:56.289604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:56.289604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:56.289604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:30:56.289604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:30:56.289604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:30:56.289604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:30:56.289604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:30:56.289604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:30:56.289604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 14:30:56.289604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:56.293186 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 432/862] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:30:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:56.303926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db118dfe-5c43-416c-b1f3-ad3f2a3ef2a3 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:56.304058 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef76a3-23b4-4b6b-99f3-aa4db2bcb6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:56.304058 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef76a3-23b4-4b6b-99f3-aa4db2bcb6d5 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:56.304058 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef76a3-23b4-4b6b-99f3-aa4db2bcb6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:56.304058 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef76a3-23b4-4b6b-99f3-aa4db2bcb6d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:56.304358 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef76a3-23b4-4b6b-99f3-aa4db2bcb6d5 None admin] RBAC: Adding query filter params (role.id=9074ca9226f8483c885f32fa868a1acc, scope.project.id=c4c8da3011294783b2180ade1bfd9366, user.id=5179c3c4eaaa41f3beb79ba7f1993478) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:56.304358 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef76a3-23b4-4b6b-99f3-aa4db2bcb6d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:56.308864 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef76a3-23b4-4b6b-99f3-aa4db2bcb6d5 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:56.326333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 431/863] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:30:56 2022] GET /identity/v3/role_assignments?user.id=5179c3c4eaaa41f3beb79ba7f1993478&scope.project.id=c4c8da3011294783b2180ade1bfd9366&role.id=9074ca9226f8483c885f32fa868a1acc => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:56.490845 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db118dfe-5c43-416c-b1f3-ad3f2a3ef2a3 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:56.539755 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0807c1-0b3f-4112-971d-f92acc33b042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:56.540770 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0807c1-0b3f-4112-971d-f92acc33b042 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:56.541262 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0807c1-0b3f-4112-971d-f92acc33b042 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:56.541746 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0807c1-0b3f-4112-971d-f92acc33b042 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:56.542794 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0807c1-0b3f-4112-971d-f92acc33b042 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:56.543334 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0807c1-0b3f-4112-971d-f92acc33b042 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:56.545061 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0807c1-0b3f-4112-971d-f92acc33b042 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:56.547632 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15485ef-0d8d-4d1c-949a-24298c3c1b13 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:56.547930 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15485ef-0d8d-4d1c-949a-24298c3c1b13 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:56.548230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15485ef-0d8d-4d1c-949a-24298c3c1b13 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:56.549739 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 432/864] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:30:56.556089 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a36f53-e43a-4802-b503-c51ccb7509ab None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:56.556419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a36f53-e43a-4802-b503-c51ccb7509ab None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:56.556684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a36f53-e43a-4802-b503-c51ccb7509ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:56.570997 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 433/865] 10.222.0.97 () {56 vars in 1135 bytes} [Tue Jul 26 14:30:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:30:56.577640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0807c1-0b3f-4112-971d-f92acc33b042 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:56.605386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-69a36f53-e43a-4802-b503-c51ccb7509ab None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:56.607533 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-69a36f53-e43a-4802-b503-c51ccb7509ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:56.633725 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:56.733782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 433/866] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:30:56.759061 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0807c1-0b3f-4112-971d-f92acc33b042 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:56.771325 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a36f53-e43a-4802-b503-c51ccb7509ab None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:56.810161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e4a4de-e818-4eb9-9fec-2ccd0e32ad0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:56.810161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4a4de-e818-4eb9-9fec-2ccd0e32ad0d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:56.811331 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4a4de-e818-4eb9-9fec-2ccd0e32ad0d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:56.812847 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4a4de-e818-4eb9-9fec-2ccd0e32ad0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:56.960417 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a36f53-e43a-4802-b503-c51ccb7509ab None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:56.961994 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e4a4de-e818-4eb9-9fec-2ccd0e32ad0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:56.963488 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e4a4de-e818-4eb9-9fec-2ccd0e32ad0d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:56.977380 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:57.005460 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:57.009294 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17952e-f8ea-49b5-bb2f-f2990f543c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:57.011520 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17952e-f8ea-49b5-bb2f-f2990f543c0f None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:57.011968 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17952e-f8ea-49b5-bb2f-f2990f543c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:57.013099 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17952e-f8ea-49b5-bb2f-f2990f543c0f None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:57.014003 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab17952e-f8ea-49b5-bb2f-f2990f543c0f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:57.016710 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab17952e-f8ea-49b5-bb2f-f2990f543c0f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:57.119302 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 434/867] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:30:56 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:30:57.122959 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 434/868] 10.222.0.97 () {60 vars in 1141 bytes} [Tue Jul 26 14:30:56 2022] POST /identity/v3/services => generated 251 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:30:57.132790 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e4a4de-e818-4eb9-9fec-2ccd0e32ad0d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:57.277626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e4a4de-e818-4eb9-9fec-2ccd0e32ad0d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:57.308014 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:57.326946 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567417b-8fb1-44e7-b7bf-49a88d025f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:57.327890 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567417b-8fb1-44e7-b7bf-49a88d025f57 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:57.328320 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567417b-8fb1-44e7-b7bf-49a88d025f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:57.328739 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567417b-8fb1-44e7-b7bf-49a88d025f57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:57.347011 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5567417b-8fb1-44e7-b7bf-49a88d025f57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:57.347011 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5567417b-8fb1-44e7-b7bf-49a88d025f57 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:57.358324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-5567417b-8fb1-44e7-b7bf-49a88d025f57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:57.358324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:57.358324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:57.358324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:57.358324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:57.358324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:57.358324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:57.358324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:57.358324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:57.358324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:57.358324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:57.358324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:57.358324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:57.358324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:57.358324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:57.359231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:57.359231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:57.359231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:57.359231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:57.359231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:57.359231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:57.359231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:57.359231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:57.359231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:57.359231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:57.359231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:57.359231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:57.359231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:57.359231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:57.359231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:57.359231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:57.359231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:57.360584 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:57.360584 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:57.360584 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:57.360584 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:57.363515 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 435/869] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:57.366323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17952e-f8ea-49b5-bb2f-f2990f543c0f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:57.503303 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17952e-f8ea-49b5-bb2f-f2990f543c0f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:57.541093 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7e192-b534-4e85-a23e-4c0b68f4cf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:57.541654 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7e192-b534-4e85-a23e-4c0b68f4cf99 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:57.541944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7e192-b534-4e85-a23e-4c0b68f4cf99 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:57.542224 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7e192-b534-4e85-a23e-4c0b68f4cf99 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:57.542924 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7e192-b534-4e85-a23e-4c0b68f4cf99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:57.543234 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7e192-b534-4e85-a23e-4c0b68f4cf99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:57.544642 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7e192-b534-4e85-a23e-4c0b68f4cf99 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:57.558906 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 435/870] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:57.571439 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567417b-8fb1-44e7-b7bf-49a88d025f57 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:57.733907 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567417b-8fb1-44e7-b7bf-49a88d025f57 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:57.775184 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fdec2e-4fa8-431a-a731-6b391579e9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:57.775889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fdec2e-4fa8-431a-a731-6b391579e9ec None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:57.776170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fdec2e-4fa8-431a-a731-6b391579e9ec None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:57.776449 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fdec2e-4fa8-431a-a731-6b391579e9ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:57.792371 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fdec2e-4fa8-431a-a731-6b391579e9ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:57.793779 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fdec2e-4fa8-431a-a731-6b391579e9ec None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:57.817052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-12fdec2e-4fa8-431a-a731-6b391579e9ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:57.817052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:57.817052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:57.817052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:57.817052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:57.817052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:57.817052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:57.817052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:57.817052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:57.817052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:57.817052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:57.817052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:57.817052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:30:57.817052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:30:57.817052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:30:57.817987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:30:57.817987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:57.817987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:57.817987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:57.817987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:57.817987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:57.817987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:57.817987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:57.817987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:57.817987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:57.817987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:57.817987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:57.817987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:57.817987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:57.817987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:57.817987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:57.817987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:57.819383 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:30:57.819383 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:30:57.819383 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:30:57.819383 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:30:57.819383 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:30:57.819383 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:30:57.819383 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:30:57.819383 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:57.819814 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 436/871] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:30:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:57.828707 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7e192-b534-4e85-a23e-4c0b68f4cf99 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:57.877646 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:57.955163 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7e192-b534-4e85-a23e-4c0b68f4cf99 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:57.993574 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c44f0d-f5b4-4129-94bc-6b0bff571ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:57.994198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c44f0d-f5b4-4129-94bc-6b0bff571ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:57.994477 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c44f0d-f5b4-4129-94bc-6b0bff571ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:57.994749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c44f0d-f5b4-4129-94bc-6b0bff571ca3 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:57.995685 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c44f0d-f5b4-4129-94bc-6b0bff571ca3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:57.996061 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c44f0d-f5b4-4129-94bc-6b0bff571ca3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:57.997639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c44f0d-f5b4-4129-94bc-6b0bff571ca3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:58.010771 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 436/872] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:30:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:58.018714 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fdec2e-4fa8-431a-a731-6b391579e9ec None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:58.098866 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:58.192632 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fdec2e-4fa8-431a-a731-6b391579e9ec None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:58.233965 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc423745-40c3-46d7-99b5-7cf7c9428b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:58.233965 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc423745-40c3-46d7-99b5-7cf7c9428b25 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:58.233965 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc423745-40c3-46d7-99b5-7cf7c9428b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:58.233965 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc423745-40c3-46d7-99b5-7cf7c9428b25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:58.234463 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc423745-40c3-46d7-99b5-7cf7c9428b25 None admin] RBAC: Adding query filter params (role.id=51b2d10dfffb4b75b186f3d8b15e2a78, scope.project.id=c4c8da3011294783b2180ade1bfd9366, user.id=85122c4052fd4ba19a2dae185a707caf) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:58.234463 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc423745-40c3-46d7-99b5-7cf7c9428b25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:58.234463 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc423745-40c3-46d7-99b5-7cf7c9428b25 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:58.242784 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 437/873] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:30:58 2022] GET /identity/v3/role_assignments?user.id=85122c4052fd4ba19a2dae185a707caf&scope.project.id=c4c8da3011294783b2180ade1bfd9366&role.id=51b2d10dfffb4b75b186f3d8b15e2a78 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:58.279748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb4ea3-99b8-493a-b1d1-52ed8b7b77a2 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:58.280402 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb4ea3-99b8-493a-b1d1-52ed8b7b77a2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:58.280890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb4ea3-99b8-493a-b1d1-52ed8b7b77a2 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:58.282797 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 437/874] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:58.291450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe136c-51a2-4237-a893-af20153f0435 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:58.291450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe136c-51a2-4237-a893-af20153f0435 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:58.291450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe136c-51a2-4237-a893-af20153f0435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:58.321736 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3cfe136c-51a2-4237-a893-af20153f0435 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:58.322937 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3cfe136c-51a2-4237-a893-af20153f0435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:58.438222 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 438/875] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:58 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:30:58.471343 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb4ea3-99b8-493a-b1d1-52ed8b7b77a2 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:58.600742 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:58.613185 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb4ea3-99b8-493a-b1d1-52ed8b7b77a2 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:58.651236 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a360c-d602-49ac-8290-e4e0e10354c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:58.652095 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a360c-d602-49ac-8290-e4e0e10354c3 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:58.652377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a360c-d602-49ac-8290-e4e0e10354c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:58.652991 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a360c-d602-49ac-8290-e4e0e10354c3 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:58.653703 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a360c-d602-49ac-8290-e4e0e10354c3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:58.655061 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a360c-d602-49ac-8290-e4e0e10354c3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:58.766634 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 438/876] 10.222.0.97 () {60 vars in 1141 bytes} [Tue Jul 26 14:30:58 2022] POST /identity/v3/services => generated 269 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:30:58.915588 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5e4ad-027c-4e3c-93d9-bf476161f935 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:58.917660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5e4ad-027c-4e3c-93d9-bf476161f935 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:58.917660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5e4ad-027c-4e3c-93d9-bf476161f935 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:58.918009 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 439/877] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:30:58.923925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4966427-e2f6-492e-9026-b5917aa6442a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:58.924216 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4966427-e2f6-492e-9026-b5917aa6442a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:58.924523 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4966427-e2f6-492e-9026-b5917aa6442a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:58.951371 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-f4966427-e2f6-492e-9026-b5917aa6442a None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:58.953591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f4966427-e2f6-492e-9026-b5917aa6442a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:59.036531 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 439/878] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:58 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:30:59.080266 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5e4ad-027c-4e3c-93d9-bf476161f935 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:59.113015 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:59.233078 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5e4ad-027c-4e3c-93d9-bf476161f935 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:59.274810 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2cfd9-737a-40e7-89ba-a4909a681208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:59.275801 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2cfd9-737a-40e7-89ba-a4909a681208 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:59.276334 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2cfd9-737a-40e7-89ba-a4909a681208 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:59.276754 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2cfd9-737a-40e7-89ba-a4909a681208 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:59.289664 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f2cfd9-737a-40e7-89ba-a4909a681208 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:30:59.294905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-35f2cfd9-737a-40e7-89ba-a4909a681208 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 14:30:59.294905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:59.294905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:59.294905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:59.294905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:59.294905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:59.294905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:59.294905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:59.294905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:59.294905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:59.294905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:59.294905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:59.294905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:30:59.294905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:30:59.294905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:30:59.294905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:30:59.296480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:30:59.296480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:30:59.296480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:30:59.296480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:30:59.296480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:59.296480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:59.296480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:59.296480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:59.296480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:59.296480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:59.296480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:59.296480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:59.296480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:59.296480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:59.296480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:59.296480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:59.297473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:59.297473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:59.297473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:59.297473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:59.297473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:30:59.297473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:30:59.297473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:30:59.297473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:30:59.297473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:30:59.297473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:30:59.297473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 14:30:59.297473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:59.304934 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 440/879] 10.222.0.97 () {56 vars in 1120 bytes} [Tue Jul 26 14:30:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:30:59.306635 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4966427-e2f6-492e-9026-b5917aa6442a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:59.424389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:59.447552 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4966427-e2f6-492e-9026-b5917aa6442a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:59.489187 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831bd15c-0582-449c-b37d-0e0d95d9a558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:59.491079 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bd15c-0582-449c-b37d-0e0d95d9a558 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:59.492039 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bd15c-0582-449c-b37d-0e0d95d9a558 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:59.502132 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bd15c-0582-449c-b37d-0e0d95d9a558 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:59.502132 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831bd15c-0582-449c-b37d-0e0d95d9a558 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:59.502132 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831bd15c-0582-449c-b37d-0e0d95d9a558 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:59.502132 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831bd15c-0582-449c-b37d-0e0d95d9a558 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:59.515309 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 440/880] 10.222.0.97 () {56 vars in 1115 bytes} [Tue Jul 26 14:30:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:30:59.517642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2cfd9-737a-40e7-89ba-a4909a681208 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:59.640127 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:59.644242 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2cfd9-737a-40e7-89ba-a4909a681208 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:59.685638 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3ceb3d-db2c-44a4-8e6a-c74bb7bc24e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:59.686677 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3ceb3d-db2c-44a4-8e6a-c74bb7bc24e3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:59.687157 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3ceb3d-db2c-44a4-8e6a-c74bb7bc24e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:59.687683 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3ceb3d-db2c-44a4-8e6a-c74bb7bc24e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:59.700758 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3ceb3d-db2c-44a4-8e6a-c74bb7bc24e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:59.702617 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3ceb3d-db2c-44a4-8e6a-c74bb7bc24e3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:59.719572 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-2a3ceb3d-db2c-44a4-8e6a-c74bb7bc24e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:59.719572 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:30:59.719572 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:30:59.719572 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:30:59.719572 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:30:59.719572 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:30:59.719572 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:30:59.719572 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:30:59.719572 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:30:59.719572 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:30:59.719572 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:30:59.719572 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:30:59.719572 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:30:59.719572 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:30:59.719572 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:30:59.720502 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:30:59.720502 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:30:59.720502 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:30:59.720502 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:30:59.720502 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:30:59.720502 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:30:59.720502 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:30:59.720502 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:30:59.720502 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:30:59.720502 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:30:59.720502 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:30:59.720502 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:30:59.720502 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:30:59.720502 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:30:59.720502 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:30:59.720502 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:30:59.720502 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:30:59.721350 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:30:59.721350 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:30:59.721350 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:30:59.721350 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:30:59.727532 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 441/881] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:30:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:30:59.729975 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831bd15c-0582-449c-b37d-0e0d95d9a558 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:59.747658 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a6edf-3ffe-4a64-bab2-a46c2890f966 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:59.748266 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a6edf-3ffe-4a64-bab2-a46c2890f966 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:59.749179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a6edf-3ffe-4a64-bab2-a46c2890f966 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:59.751357 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 442/882] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:30:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:30:59.757725 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf41364-6d4b-40df-bc95-06d7d4108599 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:59.758289 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf41364-6d4b-40df-bc95-06d7d4108599 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:59.758777 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf41364-6d4b-40df-bc95-06d7d4108599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:59.789407 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-7cf41364-6d4b-40df-bc95-06d7d4108599 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:30:59.790119 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7cf41364-6d4b-40df-bc95-06d7d4108599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:59.918259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 443/883] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:30:59 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 14:30:59.921563 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831bd15c-0582-449c-b37d-0e0d95d9a558 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:59.955571 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf41364-6d4b-40df-bc95-06d7d4108599 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:59.962784 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fcea2d-57d1-46cd-9379-9519d9bf8614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:59.963426 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fcea2d-57d1-46cd-9379-9519d9bf8614 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:59.963718 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fcea2d-57d1-46cd-9379-9519d9bf8614 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:59.963991 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fcea2d-57d1-46cd-9379-9519d9bf8614 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:59.964719 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fcea2d-57d1-46cd-9379-9519d9bf8614 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:30:59.965026 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fcea2d-57d1-46cd-9379-9519d9bf8614 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:59.966379 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fcea2d-57d1-46cd-9379-9519d9bf8614 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:59.984686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 441/884] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:30:59.994170 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fcea2d-57d1-46cd-9379-9519d9bf8614 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:00.000481 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:00.038217 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:00.081328 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf41364-6d4b-40df-bc95-06d7d4108599 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:00.142214 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28433708-6b5c-4985-a381-7cd401143350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:00.143988 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28433708-6b5c-4985-a381-7cd401143350 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:00.145032 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28433708-6b5c-4985-a381-7cd401143350 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:00.148460 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28433708-6b5c-4985-a381-7cd401143350 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:00.149337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28433708-6b5c-4985-a381-7cd401143350 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:00.150907 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28433708-6b5c-4985-a381-7cd401143350 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:00.164765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-28433708-6b5c-4985-a381-7cd401143350 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 14:31:00.164765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:00.164765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:31:00.164765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:31:00.164765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:31:00.164765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:31:00.164765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:31:00.164765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:31:00.164765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 14:31:00.164765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:00.164765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:31:00.164765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:00.164765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:00.164765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:00.164765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:00.164765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:00.164765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:00.165779 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:00.165779 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:00.165779 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:00.165779 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:00.165779 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:00.165779 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:00.165779 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:31:00.165779 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:31:00.165779 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:31:00.165779 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:31:00.165779 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:00.165779 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:00.165779 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:00.165779 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:00.165779 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:00.165779 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:00.166618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:00.166618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:00.166618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:00.166618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:00.166618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:00.166618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:00.166618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:00.166618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:00.166618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:00.166618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:00.166618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:31:00.166618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:31:00.166618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 14:31:00.166618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:00.176475 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 444/885] 10.222.0.97 () {56 vars in 1114 bytes} [Tue Jul 26 14:30:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:31:00.180295 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28433708-6b5c-4985-a381-7cd401143350 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:00.225412 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fcea2d-57d1-46cd-9379-9519d9bf8614 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:00.280422 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6cf5c-0453-40ce-b389-c0afd364b934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:00.281770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6cf5c-0453-40ce-b389-c0afd364b934 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:00.282050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6cf5c-0453-40ce-b389-c0afd364b934 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:00.282323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6cf5c-0453-40ce-b389-c0afd364b934 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:00.302416 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff6cf5c-0453-40ce-b389-c0afd364b934 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:00.303933 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff6cf5c-0453-40ce-b389-c0afd364b934 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:00.330944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-6ff6cf5c-0453-40ce-b389-c0afd364b934 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 14:31:00.330944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:00.330944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:00.330944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:00.330944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:00.330944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:00.330944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:00.330944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:00.330944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:00.330944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:00.330944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:00.330944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:00.330944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:31:00.330944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:31:00.330944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:31:00.330944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:31:00.332017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:00.332017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:00.332017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:31:00.332017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:31:00.332017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:31:00.332017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:31:00.332017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:00.332017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:00.332017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:00.332017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:00.332017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:00.332017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:00.332017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:00.332017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:00.332017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:00.332017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:00.332017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:00.332017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:00.332943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:00.332943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:00.332943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:31:00.332943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:31:00.332943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:31:00.332943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:31:00.332943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:31:00.332943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:31:00.332943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 14:31:00.332943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:00.339336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28433708-6b5c-4985-a381-7cd401143350 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:00.343986 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 442/886] 10.222.0.97 () {56 vars in 1117 bytes} [Tue Jul 26 14:30:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:31:00.352692 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6cf5c-0453-40ce-b389-c0afd364b934 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:00.376052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:00.382922 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c1ff2-11c5-4dbf-b44b-f77e85357e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:00.383957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c1ff2-11c5-4dbf-b44b-f77e85357e0e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:00.401386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c1ff2-11c5-4dbf-b44b-f77e85357e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:00.401386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c1ff2-11c5-4dbf-b44b-f77e85357e0e None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:00.401386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c1ff2-11c5-4dbf-b44b-f77e85357e0e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:00.401386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c1ff2-11c5-4dbf-b44b-f77e85357e0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:00.401386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c1ff2-11c5-4dbf-b44b-f77e85357e0e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:00.401386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 445/887] 10.222.0.97 () {56 vars in 1114 bytes} [Tue Jul 26 14:31:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:00.401877 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c1ff2-11c5-4dbf-b44b-f77e85357e0e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:00.455427 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:00.519081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6cf5c-0453-40ce-b389-c0afd364b934 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:00.559565 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b3ae4-7ad0-4f89-8727-d87da5c6556a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:00.560569 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b3ae4-7ad0-4f89-8727-d87da5c6556a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:00.561959 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b3ae4-7ad0-4f89-8727-d87da5c6556a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:00.562448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b3ae4-7ad0-4f89-8727-d87da5c6556a None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:00.563491 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b3ae4-7ad0-4f89-8727-d87da5c6556a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:00.564028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b3ae4-7ad0-4f89-8727-d87da5c6556a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:00.565962 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b3ae4-7ad0-4f89-8727-d87da5c6556a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:00.591790 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 443/888] 10.222.0.97 () {56 vars in 1149 bytes} [Tue Jul 26 14:31:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:00.593458 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b3ae4-7ad0-4f89-8727-d87da5c6556a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:00.595257 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c1ff2-11c5-4dbf-b44b-f77e85357e0e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:00.673111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fae068-501b-4678-bb31-9be5d8f368cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:00.673747 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fae068-501b-4678-bb31-9be5d8f368cc None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:00.674032 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fae068-501b-4678-bb31-9be5d8f368cc None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:00.674308 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fae068-501b-4678-bb31-9be5d8f368cc None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:00.674997 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fae068-501b-4678-bb31-9be5d8f368cc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:00.675303 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fae068-501b-4678-bb31-9be5d8f368cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:00.676702 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fae068-501b-4678-bb31-9be5d8f368cc None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:00.696065 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 446/889] 10.222.0.97 () {56 vars in 1114 bytes} [Tue Jul 26 14:31:00 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:00.701639 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fae068-501b-4678-bb31-9be5d8f368cc None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:00.735386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b3ae4-7ad0-4f89-8727-d87da5c6556a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:00.785753 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:00.814238 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d4b57-4fe8-4ace-b593-a17229cb0bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:00.816248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d4b57-4fe8-4ace-b593-a17229cb0bff None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:00.816599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d4b57-4fe8-4ace-b593-a17229cb0bff None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:00.818961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d4b57-4fe8-4ace-b593-a17229cb0bff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:00.827796 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:00.918789 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fae068-501b-4678-bb31-9be5d8f368cc None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:00.961364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d4b57-4fe8-4ace-b593-a17229cb0bff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:00.964108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d4b57-4fe8-4ace-b593-a17229cb0bff None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:00.967232 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01970a37-1c95-494d-a3da-1a9446648778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:00.967902 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01970a37-1c95-494d-a3da-1a9446648778 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:00.968191 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01970a37-1c95-494d-a3da-1a9446648778 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:00.968468 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01970a37-1c95-494d-a3da-1a9446648778 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:00.969289 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01970a37-1c95-494d-a3da-1a9446648778 None admin] RBAC: Adding query filter params (interface=public, service_id=8d4e277cff904f6881147672efdf3a89, region_id=RegionOne) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:00.969606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01970a37-1c95-494d-a3da-1a9446648778 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:00.971097 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01970a37-1c95-494d-a3da-1a9446648778 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:00.981618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 447/890] 10.222.0.97 () {56 vars in 1251 bytes} [Tue Jul 26 14:31:00 2022] GET /identity/v3/endpoints?service_id=8d4e277cff904f6881147672efdf3a89&interface=public®ion_id=RegionOne => generated 194 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:00.982999 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5043d-eecc-41a8-922c-1e7605ec977e None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:00.983309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5043d-eecc-41a8-922c-1e7605ec977e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:00.983573 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5043d-eecc-41a8-922c-1e7605ec977e None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:00.989057 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 448/891] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:00.989882 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5043d-eecc-41a8-922c-1e7605ec977e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:01.125782 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5043d-eecc-41a8-922c-1e7605ec977e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:01.131433 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 444/892] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:31:00 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 14:31:01.133267 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0dbd6-3c50-456f-bf08-df0c71587674 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:01.133770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0dbd6-3c50-456f-bf08-df0c71587674 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:01.134241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0dbd6-3c50-456f-bf08-df0c71587674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:01.154854 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:01.155622 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-36d0dbd6-3c50-456f-bf08-df0c71587674 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:01.157353 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-36d0dbd6-3c50-456f-bf08-df0c71587674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:01.169582 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc6ce1-be15-4d21-ae63-babf3b1efe9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:01.170232 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc6ce1-be15-4d21-ae63-babf3b1efe9f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:01.177360 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc6ce1-be15-4d21-ae63-babf3b1efe9f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:01.177360 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc6ce1-be15-4d21-ae63-babf3b1efe9f None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:01.177360 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc6ce1-be15-4d21-ae63-babf3b1efe9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:01.177360 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc6ce1-be15-4d21-ae63-babf3b1efe9f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:01.183640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 449/893] 10.222.0.97 () {56 vars in 1087 bytes} [Tue Jul 26 14:31:00 2022] GET /identity/v3/services => generated 1268 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:31:01.183840 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:01.185253 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc6ce1-be15-4d21-ae63-babf3b1efe9f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:01.286558 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 445/894] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:01 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:01.291829 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3a860-074d-4120-99d5-1079c77a878a None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:01.293330 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3a860-074d-4120-99d5-1079c77a878a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:01.294706 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3a860-074d-4120-99d5-1079c77a878a None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:01.300176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 446/895] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:01 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:01.310074 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351e860-f3b7-4f46-8416-eb9f812fc128 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:01.311257 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351e860-f3b7-4f46-8416-eb9f812fc128 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:01.312663 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351e860-f3b7-4f46-8416-eb9f812fc128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:01.350893 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc6ce1-be15-4d21-ae63-babf3b1efe9f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:01.353830 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a351e860-f3b7-4f46-8416-eb9f812fc128 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:01.362388 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a351e860-f3b7-4f46-8416-eb9f812fc128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:01.414270 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467181e-ab02-44ec-afd5-8b39ebf4ada5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:01.415312 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467181e-ab02-44ec-afd5-8b39ebf4ada5 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:01.415842 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467181e-ab02-44ec-afd5-8b39ebf4ada5 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:01.416317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467181e-ab02-44ec-afd5-8b39ebf4ada5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:01.433410 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3467181e-ab02-44ec-afd5-8b39ebf4ada5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:01.435172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3467181e-ab02-44ec-afd5-8b39ebf4ada5 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:01.458510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-3467181e-ab02-44ec-afd5-8b39ebf4ada5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:31:01.458510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:01.458510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:01.458510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:01.458510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:01.458510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:01.458510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:01.458510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:01.458510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:01.458510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:01.458510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:01.458510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:01.458510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:31:01.458510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:31:01.458510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:31:01.460109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:31:01.460109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:01.460109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:01.460109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:01.460109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:01.460109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:01.460109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:01.460109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:01.460109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:01.460109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:01.460109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:01.460109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:01.460109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:01.460109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:01.460109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:01.460109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:01.460109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:01.461676 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:31:01.461676 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:31:01.461676 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:31:01.461676 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:01.461676 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 450/896] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:31:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:31:01.462901 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467181e-ab02-44ec-afd5-8b39ebf4ada5 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:01.497440 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 447/897] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:01 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 14:31:01.498666 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a351e860-f3b7-4f46-8416-eb9f812fc128 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:01.560067 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:01.652986 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:01.667686 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467181e-ab02-44ec-afd5-8b39ebf4ada5 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:01.696959 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a351e860-f3b7-4f46-8416-eb9f812fc128 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:01.735257 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073aa88a-99b3-4509-9c24-850d51b7059a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:01.735936 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073aa88a-99b3-4509-9c24-850d51b7059a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:01.736215 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073aa88a-99b3-4509-9c24-850d51b7059a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:01.736496 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073aa88a-99b3-4509-9c24-850d51b7059a None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:01.737197 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073aa88a-99b3-4509-9c24-850d51b7059a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:01.737511 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073aa88a-99b3-4509-9c24-850d51b7059a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:01.747456 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073aa88a-99b3-4509-9c24-850d51b7059a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:01.759037 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 448/898] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:01.759966 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073aa88a-99b3-4509-9c24-850d51b7059a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:01.768628 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8083ae-65fd-413c-8983-00d81297ba67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:01.770576 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8083ae-65fd-413c-8983-00d81297ba67 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:01.771096 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8083ae-65fd-413c-8983-00d81297ba67 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:01.771620 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8083ae-65fd-413c-8983-00d81297ba67 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:01.772500 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8083ae-65fd-413c-8983-00d81297ba67 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:01.774093 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8083ae-65fd-413c-8983-00d81297ba67 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:01.791154 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-be8083ae-65fd-413c-8983-00d81297ba67 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 14:31:01.791154 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:01.791154 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:31:01.791154 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:31:01.791154 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:31:01.791154 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:31:01.791154 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:31:01.791154 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:31:01.791154 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 14:31:01.791154 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:01.791154 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:31:01.791154 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:01.791154 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:01.791154 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:01.791154 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:01.791154 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:01.791154 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:01.792177 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:01.792177 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:01.792177 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:01.792177 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:01.792177 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:01.792177 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:01.792177 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:31:01.792177 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:31:01.792177 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:31:01.792177 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:31:01.792177 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:01.792177 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:01.792177 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:01.792177 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:01.792177 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:01.792177 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:01.792993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:01.792993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:01.792993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:01.792993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:01.792993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:01.792993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:01.792993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:01.792993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:01.792993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:01.792993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:01.792993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:31:01.792993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:31:01.792993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 14:31:01.792993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:01.795442 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 451/899] 10.222.0.97 () {56 vars in 1126 bytes} [Tue Jul 26 14:31:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:31:01.796446 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8083ae-65fd-413c-8983-00d81297ba67 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:01.920122 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073aa88a-99b3-4509-9c24-850d51b7059a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:01.961530 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa02f74-bd9b-4258-a110-ccaaa3d5bd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:01.962343 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa02f74-bd9b-4258-a110-ccaaa3d5bd8b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:01.962728 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa02f74-bd9b-4258-a110-ccaaa3d5bd8b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:01.963109 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa02f74-bd9b-4258-a110-ccaaa3d5bd8b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:01.963894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa02f74-bd9b-4258-a110-ccaaa3d5bd8b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:01.965344 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa02f74-bd9b-4258-a110-ccaaa3d5bd8b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:01.976838 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-6aa02f74-bd9b-4258-a110-ccaaa3d5bd8b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 14:31:01.976838 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:01.976838 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:31:01.976838 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:31:01.976838 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:31:01.976838 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:31:01.976838 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:31:01.976838 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:31:01.976838 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 14:31:01.976838 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:01.976838 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:31:01.976838 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:01.976838 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:01.976838 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:01.976838 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:01.976838 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:01.977952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:01.977952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:01.977952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:01.977952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:01.977952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:01.977952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:01.977952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:01.977952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:31:01.977952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:31:01.977952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:31:01.977952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:31:01.977952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:01.977952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:01.977952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:01.977952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:01.977952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:01.978874 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:01.978874 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:01.978874 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:01.978874 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:01.978874 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:01.978874 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:01.978874 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:01.978874 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:01.978874 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:01.978874 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:01.978874 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:01.978874 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:31:01.978874 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:31:01.978874 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 14:31:01.978874 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:01.980136 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 449/900] 10.222.0.97 () {56 vars in 1132 bytes} [Tue Jul 26 14:31:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:31:01.980850 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa02f74-bd9b-4258-a110-ccaaa3d5bd8b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:01.986190 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:01.992617 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8083ae-65fd-413c-8983-00d81297ba67 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:01.999926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:02.041752 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56304e5e-d9bb-4faf-9b98-da84a904fc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:02.043005 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56304e5e-d9bb-4faf-9b98-da84a904fc45 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:02.043836 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56304e5e-d9bb-4faf-9b98-da84a904fc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:02.045768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56304e5e-d9bb-4faf-9b98-da84a904fc45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:02.065562 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56304e5e-d9bb-4faf-9b98-da84a904fc45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:02.067042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56304e5e-d9bb-4faf-9b98-da84a904fc45 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:02.080207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-56304e5e-d9bb-4faf-9b98-da84a904fc45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:31:02.080207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:02.080207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:02.080207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:02.080207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:02.080207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:02.080207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:02.080207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:02.080207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:02.080207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:02.080207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:02.080207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:02.080207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:31:02.080207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:31:02.080207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:31:02.081159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:31:02.081159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:02.081159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:02.081159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:02.081159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:02.081159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:02.081159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:02.081159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:02.081159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:02.081159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:02.081159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:02.081159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:02.081159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:02.081159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:02.081159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:02.081159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:02.081159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:02.082053 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:31:02.082053 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:31:02.082053 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:31:02.082053 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:31:02.082053 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:31:02.082053 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:31:02.082053 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:31:02.082053 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:02.082053 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 452/901] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:31:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:31:02.083280 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56304e5e-d9bb-4faf-9b98-da84a904fc45 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:02.159944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa02f74-bd9b-4258-a110-ccaaa3d5bd8b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:02.210048 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8dba1d-1061-4627-b0e0-a22b59a4b4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:02.210606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8dba1d-1061-4627-b0e0-a22b59a4b4f7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:02.211191 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8dba1d-1061-4627-b0e0-a22b59a4b4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:02.211485 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8dba1d-1061-4627-b0e0-a22b59a4b4f7 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:02.212181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8dba1d-1061-4627-b0e0-a22b59a4b4f7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:02.212476 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8dba1d-1061-4627-b0e0-a22b59a4b4f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:02.213815 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8dba1d-1061-4627-b0e0-a22b59a4b4f7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:02.221081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 450/902] 10.222.0.97 () {56 vars in 1122 bytes} [Tue Jul 26 14:31:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:02.221912 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8dba1d-1061-4627-b0e0-a22b59a4b4f7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:02.255127 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56304e5e-d9bb-4faf-9b98-da84a904fc45 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:02.304523 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe68ee-b3e1-4c9b-8432-736dd23cc25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:02.305137 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe68ee-b3e1-4c9b-8432-736dd23cc25c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:02.305482 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe68ee-b3e1-4c9b-8432-736dd23cc25c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:02.305754 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe68ee-b3e1-4c9b-8432-736dd23cc25c None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:02.306419 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfe68ee-b3e1-4c9b-8432-736dd23cc25c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:02.306725 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfe68ee-b3e1-4c9b-8432-736dd23cc25c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:02.308101 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfe68ee-b3e1-4c9b-8432-736dd23cc25c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:02.313153 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 453/903] 10.222.0.97 () {56 vars in 1126 bytes} [Tue Jul 26 14:31:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:02.313906 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe68ee-b3e1-4c9b-8432-736dd23cc25c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:02.324003 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:02.344673 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:02.365375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8dba1d-1061-4627-b0e0-a22b59a4b4f7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:02.401259 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d9608-84bf-41cc-b360-c484ac36997e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:02.401902 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d9608-84bf-41cc-b360-c484ac36997e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:02.402159 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d9608-84bf-41cc-b360-c484ac36997e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:02.402543 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d9608-84bf-41cc-b360-c484ac36997e None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:02.403209 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d9608-84bf-41cc-b360-c484ac36997e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:02.403671 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d9608-84bf-41cc-b360-c484ac36997e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:02.405158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d9608-84bf-41cc-b360-c484ac36997e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:02.418579 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 451/904] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:31:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:02.419543 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d9608-84bf-41cc-b360-c484ac36997e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:02.430937 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe68ee-b3e1-4c9b-8432-736dd23cc25c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:02.477045 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b99159-9fc3-48f1-ad5f-adc0c256cf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:02.477732 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b99159-9fc3-48f1-ad5f-adc0c256cf32 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:02.478087 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b99159-9fc3-48f1-ad5f-adc0c256cf32 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:02.478426 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b99159-9fc3-48f1-ad5f-adc0c256cf32 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:02.479173 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b99159-9fc3-48f1-ad5f-adc0c256cf32 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:02.479566 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b99159-9fc3-48f1-ad5f-adc0c256cf32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:02.481010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b99159-9fc3-48f1-ad5f-adc0c256cf32 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:02.486250 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 454/905] 10.222.0.97 () {56 vars in 1122 bytes} [Tue Jul 26 14:31:02 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:02.487103 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b99159-9fc3-48f1-ad5f-adc0c256cf32 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:02.593148 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d9608-84bf-41cc-b360-c484ac36997e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:02.614690 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:02.628647 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b99159-9fc3-48f1-ad5f-adc0c256cf32 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:02.642786 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:02.655165 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0853f105-7a15-49d4-9d42-5efc38b593eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:02.657132 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853f105-7a15-49d4-9d42-5efc38b593eb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:02.657132 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853f105-7a15-49d4-9d42-5efc38b593eb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:02.657132 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853f105-7a15-49d4-9d42-5efc38b593eb None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:02.657441 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0853f105-7a15-49d4-9d42-5efc38b593eb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:02.657783 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0853f105-7a15-49d4-9d42-5efc38b593eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:02.660551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0853f105-7a15-49d4-9d42-5efc38b593eb None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:02.665503 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 452/906] 10.222.0.97 () {56 vars in 1126 bytes} [Tue Jul 26 14:31:02 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:02.674177 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0853f105-7a15-49d4-9d42-5efc38b593eb None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:02.679618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b64be-e9eb-4749-9b86-08eb34885159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:02.680374 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b64be-e9eb-4749-9b86-08eb34885159 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:02.680728 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b64be-e9eb-4749-9b86-08eb34885159 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:02.681008 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b64be-e9eb-4749-9b86-08eb34885159 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:02.681798 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b64be-e9eb-4749-9b86-08eb34885159 None admin] RBAC: Adding query filter params (role.id=a61134ee45dd49b3a20a8a55044e0e3c, scope.project.id=c4c8da3011294783b2180ade1bfd9366, user.id=5179c3c4eaaa41f3beb79ba7f1993478) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:02.682108 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b64be-e9eb-4749-9b86-08eb34885159 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:02.686002 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b64be-e9eb-4749-9b86-08eb34885159 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:02.692487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 455/907] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:31:02 2022] GET /identity/v3/role_assignments?user.id=5179c3c4eaaa41f3beb79ba7f1993478&scope.project.id=c4c8da3011294783b2180ade1bfd9366&role.id=a61134ee45dd49b3a20a8a55044e0e3c => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:02.806593 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0853f105-7a15-49d4-9d42-5efc38b593eb None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:02.846664 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee4863-8588-46d6-b520-bcdee2e80588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:02.847303 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee4863-8588-46d6-b520-bcdee2e80588 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:02.847701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee4863-8588-46d6-b520-bcdee2e80588 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:02.848011 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee4863-8588-46d6-b520-bcdee2e80588 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:02.848813 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee4863-8588-46d6-b520-bcdee2e80588 None admin] RBAC: Adding query filter params (interface=public, service_id=d565d389b87a49c9b4e82a7b635204e0, region_id=RegionOne) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:02.849666 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee4863-8588-46d6-b520-bcdee2e80588 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:02.851426 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee4863-8588-46d6-b520-bcdee2e80588 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:02.858669 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 453/908] 10.222.0.97 () {56 vars in 1251 bytes} [Tue Jul 26 14:31:02 2022] GET /identity/v3/endpoints?service_id=d565d389b87a49c9b4e82a7b635204e0&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:02.868581 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b64be-e9eb-4749-9b86-08eb34885159 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:02.977707 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b64be-e9eb-4749-9b86-08eb34885159 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:03.013342 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de719d-8e1d-48e0-83cb-5f3edd155328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:03.013962 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de719d-8e1d-48e0-83cb-5f3edd155328 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:03.014534 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de719d-8e1d-48e0-83cb-5f3edd155328 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:03.014808 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de719d-8e1d-48e0-83cb-5f3edd155328 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:03.015489 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36de719d-8e1d-48e0-83cb-5f3edd155328 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:03.016870 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36de719d-8e1d-48e0-83cb-5f3edd155328 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:03.023661 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 456/909] 10.222.0.97 () {56 vars in 1087 bytes} [Tue Jul 26 14:31:02 2022] GET /identity/v3/services => generated 1268 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:31:03.354784 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcf810-2181-4df6-b0e2-a779dfcb7674 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:03.355166 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcf810-2181-4df6-b0e2-a779dfcb7674 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:03.355350 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcf810-2181-4df6-b0e2-a779dfcb7674 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:03.356962 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 454/910] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:03.372939 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e131f-20bb-432b-a31c-f5b3f984ef2e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:03.372939 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e131f-20bb-432b-a31c-f5b3f984ef2e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:03.372939 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e131f-20bb-432b-a31c-f5b3f984ef2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:03.391207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-2b4e131f-20bb-432b-a31c-f5b3f984ef2e None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:03.392965 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2b4e131f-20bb-432b-a31c-f5b3f984ef2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:03.409048 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:03.468450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 457/911] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:03.493893 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdcf810-2181-4df6-b0e2-a779dfcb7674 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:03.592691 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:03.655507 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdcf810-2181-4df6-b0e2-a779dfcb7674 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:03.696759 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9518c6-3150-4e7a-9787-79a69885b63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:03.698067 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9518c6-3150-4e7a-9787-79a69885b63f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:03.698067 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9518c6-3150-4e7a-9787-79a69885b63f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:03.698067 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9518c6-3150-4e7a-9787-79a69885b63f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:03.698420 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9518c6-3150-4e7a-9787-79a69885b63f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:03.701471 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9518c6-3150-4e7a-9787-79a69885b63f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:03.713820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-fc9518c6-3150-4e7a-9787-79a69885b63f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 14:31:03.713820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:03.713820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:31:03.713820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:31:03.713820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:31:03.713820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:31:03.713820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:31:03.713820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:31:03.713820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 14:31:03.713820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:03.713820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:31:03.713820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:03.713820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:03.713820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:03.713820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:03.713820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:03.713820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:03.714817 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:03.714817 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:03.714817 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:03.714817 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:03.714817 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:03.714817 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:03.714817 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:31:03.714817 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:31:03.714817 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:31:03.714817 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:31:03.714817 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:03.714817 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:03.714817 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:03.714817 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:03.714817 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:03.714817 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:03.715730 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:03.715730 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:03.715730 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:03.715730 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:03.715730 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:03.715730 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:03.715730 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:03.715730 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:03.715730 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:03.715730 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:03.715730 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:31:03.715730 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:31:03.715730 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 14:31:03.715730 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:03.715730 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 455/912] 10.222.0.97 () {56 vars in 1114 bytes} [Tue Jul 26 14:31:03 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:31:03.725551 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e131f-20bb-432b-a31c-f5b3f984ef2e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:03.836298 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e131f-20bb-432b-a31c-f5b3f984ef2e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:03.878574 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18fa605-008f-4a81-88c2-b4b824f1b26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:03.879587 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18fa605-008f-4a81-88c2-b4b824f1b26a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:03.880076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18fa605-008f-4a81-88c2-b4b824f1b26a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:03.881104 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18fa605-008f-4a81-88c2-b4b824f1b26a None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:03.882029 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18fa605-008f-4a81-88c2-b4b824f1b26a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:03.882534 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18fa605-008f-4a81-88c2-b4b824f1b26a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:03.884192 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18fa605-008f-4a81-88c2-b4b824f1b26a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:03.891347 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 458/913] 10.222.0.97 () {56 vars in 1114 bytes} [Tue Jul 26 14:31:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:03.897363 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9518c6-3150-4e7a-9787-79a69885b63f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:04.057880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9518c6-3150-4e7a-9787-79a69885b63f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:04.107835 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:04.108863 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c14625-d7bb-491c-b152-2ed7bc957458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:04.109384 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c14625-d7bb-491c-b152-2ed7bc957458 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:04.109939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c14625-d7bb-491c-b152-2ed7bc957458 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:04.110226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c14625-d7bb-491c-b152-2ed7bc957458 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:04.110908 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c14625-d7bb-491c-b152-2ed7bc957458 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:04.111219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c14625-d7bb-491c-b152-2ed7bc957458 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:04.112654 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c14625-d7bb-491c-b152-2ed7bc957458 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:04.118588 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 456/914] 10.222.0.97 () {56 vars in 1114 bytes} [Tue Jul 26 14:31:03 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:04.124476 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18fa605-008f-4a81-88c2-b4b824f1b26a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:04.142355 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:04.238237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18fa605-008f-4a81-88c2-b4b824f1b26a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:04.280589 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04ff89-8834-4237-8452-4b12156ee3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:04.281593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04ff89-8834-4237-8452-4b12156ee3d4 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:04.282110 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04ff89-8834-4237-8452-4b12156ee3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:04.282604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04ff89-8834-4237-8452-4b12156ee3d4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:04.283571 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be04ff89-8834-4237-8452-4b12156ee3d4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:04.285196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be04ff89-8834-4237-8452-4b12156ee3d4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:04.390660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 459/915] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:31:04 2022] POST /identity/v3/endpoints => generated 346 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:31:04.669583 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c67452-6c4c-4d1e-a4d6-69ecbd742ae0 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:04.670100 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c67452-6c4c-4d1e-a4d6-69ecbd742ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:04.670100 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c67452-6c4c-4d1e-a4d6-69ecbd742ae0 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:04.671758 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 457/916] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:04.677392 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ab90a-f434-4287-855c-ab19339f1cba None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:04.677867 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ab90a-f434-4287-855c-ab19339f1cba None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:04.677983 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ab90a-f434-4287-855c-ab19339f1cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:04.700956 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ad0ab90a-f434-4287-855c-ab19339f1cba None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:04.702670 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ad0ab90a-f434-4287-855c-ab19339f1cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:04.788386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:04.803534 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 460/917] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:04.822372 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c67452-6c4c-4d1e-a4d6-69ecbd742ae0 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:04.936729 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc522f-0ad7-40a2-a3d9-bb0aa4d9638a None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:04.936975 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc522f-0ad7-40a2-a3d9-bb0aa4d9638a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:04.937223 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc522f-0ad7-40a2-a3d9-bb0aa4d9638a None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:04.938738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 461/918] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:31:04.945495 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2dc55-98e7-45aa-b0c9-9d525625eff7 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:04.945810 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2dc55-98e7-45aa-b0c9-9d525625eff7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:04.946104 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2dc55-98e7-45aa-b0c9-9d525625eff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:04.965054 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-20e2dc55-98e7-45aa-b0c9-9d525625eff7 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:04.966461 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-20e2dc55-98e7-45aa-b0c9-9d525625eff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:04.976293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c67452-6c4c-4d1e-a4d6-69ecbd742ae0 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:05.031700 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c67ba2f-c561-4e5e-aee1-ee9f314c56a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:05.033178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67ba2f-c561-4e5e-aee1-ee9f314c56a8 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:05.038490 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67ba2f-c561-4e5e-aee1-ee9f314c56a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:05.039104 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67ba2f-c561-4e5e-aee1-ee9f314c56a8 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:05.040054 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c67ba2f-c561-4e5e-aee1-ee9f314c56a8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:05.041971 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c67ba2f-c561-4e5e-aee1-ee9f314c56a8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:05.064455 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 462/919] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 14:31:05.088004 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2dc55-98e7-45aa-b0c9-9d525625eff7 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:05.163193 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 458/920] 10.222.0.97 () {60 vars in 1141 bytes} [Tue Jul 26 14:31:04 2022] POST /identity/v3/services => generated 241 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:31:05.206922 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2dc55-98e7-45aa-b0c9-9d525625eff7 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:05.233147 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:05.253712 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ce5f5-718d-45b8-a1c0-aec7bbc3beaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:05.254746 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ce5f5-718d-45b8-a1c0-aec7bbc3beaf None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:05.254746 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ce5f5-718d-45b8-a1c0-aec7bbc3beaf None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:05.255846 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ce5f5-718d-45b8-a1c0-aec7bbc3beaf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:05.268899 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ce5f5-718d-45b8-a1c0-aec7bbc3beaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:31:05.274794 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-2c1ce5f5-718d-45b8-a1c0-aec7bbc3beaf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 14:31:05.274794 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:05.274794 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:05.274794 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:05.274794 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:05.274794 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:05.274794 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:05.274794 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:05.274794 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:05.274794 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:05.274794 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:05.274794 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:05.274794 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:31:05.274794 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:31:05.274794 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:31:05.274794 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:31:05.275802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:31:05.275802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:31:05.275802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:31:05.275802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:31:05.275802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:05.275802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:05.275802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:05.275802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:05.275802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:05.275802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:05.275802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:05.275802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:05.275802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:05.275802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:05.275802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:05.275802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:05.276642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:05.276642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:05.276642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:05.276642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:05.276642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:31:05.276642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:31:05.276642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:31:05.276642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:31:05.276642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:31:05.276642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:31:05.276642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 14:31:05.276642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:05.276642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 463/921] 10.222.0.97 () {56 vars in 1120 bytes} [Tue Jul 26 14:31:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:31:05.282947 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c67ba2f-c561-4e5e-aee1-ee9f314c56a8 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:05.294247 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9914b7-1901-4b28-9aee-0471b08734da None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:05.295466 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9914b7-1901-4b28-9aee-0471b08734da None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:05.295466 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9914b7-1901-4b28-9aee-0471b08734da None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:05.297930 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 464/922] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:05.307343 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8074774-81f1-4cf9-8b75-9290ee812dc0 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:05.307768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8074774-81f1-4cf9-8b75-9290ee812dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:05.308001 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8074774-81f1-4cf9-8b75-9290ee812dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:05.331865 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f8074774-81f1-4cf9-8b75-9290ee812dc0 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:05.333099 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f8074774-81f1-4cf9-8b75-9290ee812dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:05.353704 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:05.440900 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 465/923] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:05.449297 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c67ba2f-c561-4e5e-aee1-ee9f314c56a8 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:05.464905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8074774-81f1-4cf9-8b75-9290ee812dc0 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:05.492236 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c175eb-06d3-4450-b83e-32f758f1ac05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:05.493513 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c175eb-06d3-4450-b83e-32f758f1ac05 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:05.494525 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c175eb-06d3-4450-b83e-32f758f1ac05 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:05.495252 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c175eb-06d3-4450-b83e-32f758f1ac05 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:05.496287 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c175eb-06d3-4450-b83e-32f758f1ac05 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:05.497170 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c175eb-06d3-4450-b83e-32f758f1ac05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:05.499550 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c175eb-06d3-4450-b83e-32f758f1ac05 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:05.516060 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 459/924] 10.222.0.97 () {56 vars in 1115 bytes} [Tue Jul 26 14:31:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:05.518431 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c175eb-06d3-4450-b83e-32f758f1ac05 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:05.585464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8074774-81f1-4cf9-8b75-9290ee812dc0 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:05.599909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:05.639337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eabce1-40a7-4af0-9064-39e240e68086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:05.640929 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eabce1-40a7-4af0-9064-39e240e68086 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:05.641450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eabce1-40a7-4af0-9064-39e240e68086 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:05.642083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eabce1-40a7-4af0-9064-39e240e68086 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:05.643013 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eabce1-40a7-4af0-9064-39e240e68086 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:05.646161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eabce1-40a7-4af0-9064-39e240e68086 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:05.659605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c175eb-06d3-4450-b83e-32f758f1ac05 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:05.660480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-29eabce1-40a7-4af0-9064-39e240e68086 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 14:31:05.660480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:05.660480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:31:05.660480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:31:05.660480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:31:05.660480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:31:05.660480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:31:05.660480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:31:05.660480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 14:31:05.660480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:05.660480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:31:05.660480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:05.660480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:05.660480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:05.660480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:05.660480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:05.661764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:05.661764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:05.661764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:05.661764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:05.661764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:05.661764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:05.661764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:05.661764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:31:05.661764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:31:05.661764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:31:05.661764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:31:05.661764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:05.661764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:05.661764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:05.661764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:05.661764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:05.662599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:05.662599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:05.662599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:05.662599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:05.662599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:05.662599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:05.662599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:05.662599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:05.662599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:05.662599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:05.662599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:05.662599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:31:05.662599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:31:05.662599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 14:31:05.662599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:05.662599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 466/925] 10.222.0.97 () {56 vars in 1132 bytes} [Tue Jul 26 14:31:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:31:05.669470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eabce1-40a7-4af0-9064-39e240e68086 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:05.674783 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:05.712225 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4722d39c-0c1d-4831-818d-440c0680c079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:05.713327 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4722d39c-0c1d-4831-818d-440c0680c079 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:05.713951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4722d39c-0c1d-4831-818d-440c0680c079 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:05.714595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4722d39c-0c1d-4831-818d-440c0680c079 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:05.729654 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4722d39c-0c1d-4831-818d-440c0680c079 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:05.732061 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4722d39c-0c1d-4831-818d-440c0680c079 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:05.753198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-4722d39c-0c1d-4831-818d-440c0680c079 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:31:05.753198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:05.753198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:05.753198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:05.753198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:05.753198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:05.753198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:05.753198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:05.753198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:05.753198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:05.753198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:05.753198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:05.753198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:31:05.753198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:31:05.753198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:31:05.754135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:31:05.754135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:05.754135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:05.754135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:05.754135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:05.754135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:05.754135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:05.754135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:05.754135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:05.754135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:05.754135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:05.754135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:05.754135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:05.754135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:05.754135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:05.754135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:05.754135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:05.754996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:31:05.754996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:31:05.754996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:31:05.754996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:05.757367 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 460/926] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:31:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:31:05.767500 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4722d39c-0c1d-4831-818d-440c0680c079 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:05.804558 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eabce1-40a7-4af0-9064-39e240e68086 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:05.847994 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d369b-c222-4450-9c19-1c8a91c7f792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:05.849338 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d369b-c222-4450-9c19-1c8a91c7f792 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:05.849654 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d369b-c222-4450-9c19-1c8a91c7f792 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:05.849928 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d369b-c222-4450-9c19-1c8a91c7f792 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:05.850618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3d369b-c222-4450-9c19-1c8a91c7f792 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:05.851046 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3d369b-c222-4450-9c19-1c8a91c7f792 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:05.853016 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3d369b-c222-4450-9c19-1c8a91c7f792 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:05.859575 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 467/927] 10.222.0.97 () {56 vars in 1126 bytes} [Tue Jul 26 14:31:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:05.865865 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d369b-c222-4450-9c19-1c8a91c7f792 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:05.910668 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4722d39c-0c1d-4831-818d-440c0680c079 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:05.918797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:05.956627 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb0274-a2b8-469b-a778-feecab17bc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:05.957316 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb0274-a2b8-469b-a778-feecab17bc8a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:05.957610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb0274-a2b8-469b-a778-feecab17bc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:05.957892 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb0274-a2b8-469b-a778-feecab17bc8a None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:05.959590 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb0274-a2b8-469b-a778-feecab17bc8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:05.959924 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb0274-a2b8-469b-a778-feecab17bc8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:05.961311 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb0274-a2b8-469b-a778-feecab17bc8a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:05.978647 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 461/928] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:31:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:05.984822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb0274-a2b8-469b-a778-feecab17bc8a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:06.016760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d369b-c222-4450-9c19-1c8a91c7f792 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:06.020645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:06.072458 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b91cd8-cbdb-42ed-8662-ea467c8bdddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:06.073893 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b91cd8-cbdb-42ed-8662-ea467c8bdddc None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:06.074643 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b91cd8-cbdb-42ed-8662-ea467c8bdddc None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:06.075110 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b91cd8-cbdb-42ed-8662-ea467c8bdddc None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:06.076234 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b91cd8-cbdb-42ed-8662-ea467c8bdddc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:06.076734 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b91cd8-cbdb-42ed-8662-ea467c8bdddc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:06.078859 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b91cd8-cbdb-42ed-8662-ea467c8bdddc None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:06.086253 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 468/929] 10.222.0.97 () {56 vars in 1126 bytes} [Tue Jul 26 14:31:05 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:06.092246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b91cd8-cbdb-42ed-8662-ea467c8bdddc None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:06.135037 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb0274-a2b8-469b-a778-feecab17bc8a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:06.190880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae208a-6f8e-4c10-867b-addc0e788a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:06.192055 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae208a-6f8e-4c10-867b-addc0e788a8e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:06.192493 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae208a-6f8e-4c10-867b-addc0e788a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:06.193559 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae208a-6f8e-4c10-867b-addc0e788a8e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:06.204052 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae208a-6f8e-4c10-867b-addc0e788a8e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:06.206671 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae208a-6f8e-4c10-867b-addc0e788a8e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:06.215069 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b91cd8-cbdb-42ed-8662-ea467c8bdddc None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:06.217049 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:06.224691 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-82ae208a-6f8e-4c10-867b-addc0e788a8e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 14:31:06.224691 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:06.224691 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:06.224691 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:06.224691 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:06.224691 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:06.224691 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:06.224691 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:06.224691 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:06.224691 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:06.224691 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:06.224691 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:06.224691 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:31:06.224691 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:31:06.224691 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:31:06.224691 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:31:06.227953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:06.227953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:06.227953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:31:06.227953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:31:06.227953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:31:06.227953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:31:06.227953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:06.227953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:06.227953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:06.227953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:06.227953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:06.227953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:06.227953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:06.227953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:06.227953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:06.227953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:06.227953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:06.227953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:06.228917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:06.228917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:06.228917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:31:06.228917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:31:06.228917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:31:06.228917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:31:06.228917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:31:06.228917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:31:06.228917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 14:31:06.228917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:06.231676 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 462/930] 10.222.0.97 () {56 vars in 1117 bytes} [Tue Jul 26 14:31:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:31:06.232651 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae208a-6f8e-4c10-867b-addc0e788a8e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:06.274108 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1530f4e-8d21-47d3-aa04-d8b0feef9487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:06.275011 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1530f4e-8d21-47d3-aa04-d8b0feef9487 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:06.275308 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1530f4e-8d21-47d3-aa04-d8b0feef9487 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:06.275653 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1530f4e-8d21-47d3-aa04-d8b0feef9487 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:06.276983 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1530f4e-8d21-47d3-aa04-d8b0feef9487 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:06.278640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1530f4e-8d21-47d3-aa04-d8b0feef9487 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:06.335876 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:06.359326 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae208a-6f8e-4c10-867b-addc0e788a8e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:06.407732 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 469/931] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:31:06 2022] POST /identity/v3/endpoints => generated 347 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:31:06.424520 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2077d-60c5-4f80-a663-aedc2c907e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:06.425644 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2077d-60c5-4f80-a663-aedc2c907e98 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:06.426126 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2077d-60c5-4f80-a663-aedc2c907e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:06.426588 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2077d-60c5-4f80-a663-aedc2c907e98 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:06.428730 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2077d-60c5-4f80-a663-aedc2c907e98 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:06.430416 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2077d-60c5-4f80-a663-aedc2c907e98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:06.431300 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2077d-60c5-4f80-a663-aedc2c907e98 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:06.469559 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 463/932] 10.222.0.97 () {56 vars in 1149 bytes} [Tue Jul 26 14:31:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:06.475203 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1530f4e-8d21-47d3-aa04-d8b0feef9487 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:06.597561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1530f4e-8d21-47d3-aa04-d8b0feef9487 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:06.615758 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e475c0b-c968-4274-9d20-ce7a94e40b52 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:06.616055 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e475c0b-c968-4274-9d20-ce7a94e40b52 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:06.616304 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e475c0b-c968-4274-9d20-ce7a94e40b52 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:06.617929 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 464/933] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:31:06.622733 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b91238-32dc-4f17-a297-cf7c2c63052b None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:06.623018 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b91238-32dc-4f17-a297-cf7c2c63052b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:06.623289 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b91238-32dc-4f17-a297-cf7c2c63052b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:06.637335 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a930c-412c-43bb-874c-0ea76fb5085f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:06.637983 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a930c-412c-43bb-874c-0ea76fb5085f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:06.638360 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a930c-412c-43bb-874c-0ea76fb5085f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:06.638546 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a930c-412c-43bb-874c-0ea76fb5085f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:06.643381 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-56b91238-32dc-4f17-a297-cf7c2c63052b None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:06.644878 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-56b91238-32dc-4f17-a297-cf7c2c63052b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:06.650467 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a930c-412c-43bb-874c-0ea76fb5085f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:06.652039 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a930c-412c-43bb-874c-0ea76fb5085f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:06.672388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-aa7a930c-412c-43bb-874c-0ea76fb5085f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:31:06.672388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:06.672388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:06.672388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:06.672388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:06.672388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:06.672388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:06.672388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:06.672388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:06.672388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:06.672388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:06.672388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:06.672388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:31:06.672388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:31:06.672388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:31:06.673315 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:31:06.673315 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:06.673315 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:06.673315 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:06.673315 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:06.673315 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:06.673315 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:06.673315 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:06.673315 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:06.673315 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:06.673315 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:06.673315 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:06.673315 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:06.673315 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:06.673315 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:06.673315 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:06.673315 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:06.674192 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:31:06.674192 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:31:06.674192 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:31:06.674192 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:06.674842 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 470/934] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:31:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:31:06.680453 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a930c-412c-43bb-874c-0ea76fb5085f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:06.703788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:06.730955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 465/935] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:06.756480 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b91238-32dc-4f17-a297-cf7c2c63052b None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:06.832728 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a930c-412c-43bb-874c-0ea76fb5085f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:06.841599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:06.882428 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97c21e-3f67-4e4e-8eb3-2cfc6c6ada18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:06.883441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97c21e-3f67-4e4e-8eb3-2cfc6c6ada18 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:06.883841 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97c21e-3f67-4e4e-8eb3-2cfc6c6ada18 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:06.884286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97c21e-3f67-4e4e-8eb3-2cfc6c6ada18 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:06.885440 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c97c21e-3f67-4e4e-8eb3-2cfc6c6ada18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:06.886266 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c97c21e-3f67-4e4e-8eb3-2cfc6c6ada18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:06.888113 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c97c21e-3f67-4e4e-8eb3-2cfc6c6ada18 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:06.889534 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b91238-32dc-4f17-a297-cf7c2c63052b None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:06.905841 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 471/936] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:06.911161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97c21e-3f67-4e4e-8eb3-2cfc6c6ada18 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:06.942150 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40edeba2-9ef7-4bc8-96b6-b2c2a0ab1e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:06.943739 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40edeba2-9ef7-4bc8-96b6-b2c2a0ab1e91 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:06.944252 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40edeba2-9ef7-4bc8-96b6-b2c2a0ab1e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:06.944739 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40edeba2-9ef7-4bc8-96b6-b2c2a0ab1e91 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:06.945613 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40edeba2-9ef7-4bc8-96b6-b2c2a0ab1e91 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:06.947265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40edeba2-9ef7-4bc8-96b6-b2c2a0ab1e91 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:06.950140 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: WARNING keystone.server.flask.common [None req-40edeba2-9ef7-4bc8-96b6-b2c2a0ab1e91 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 26 14:31:06.997366 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:07.021389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.sql.core [None req-40edeba2-9ef7-4bc8-96b6-b2c2a0ab1e91 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 26 14:31:07.021389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 26 14:31:07.021389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [parameters: {'id': '0411764ec366422aa5e65f9663c2630b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 26 14:31:07.021389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80523) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 26 14:31:07.028240 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97c21e-3f67-4e4e-8eb3-2cfc6c6ada18 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:07.030459 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-40edeba2-9ef7-4bc8-96b6-b2c2a0ab1e91 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0411764ec366422aa5e65f9663c2630b.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0411764ec366422aa5e65f9663c2630b. Jul 26 14:31:07.030459 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:07.030459 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 14:31:07.030459 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 14:31:07.030459 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 14:31:07.030459 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 14:31:07.030459 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 14:31:07.030459 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 14:31:07.030459 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 14:31:07.030459 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application conn.query(q) Jul 26 14:31:07.030459 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 14:31:07.030459 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 14:31:07.030459 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 14:31:07.031536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application result.read() Jul 26 14:31:07.031536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 14:31:07.031536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 14:31:07.031536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 14:31:07.031536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 14:31:07.031536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 14:31:07.031536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 14:31:07.031536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 14:31:07.031536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 14:31:07.031536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 26 14:31:07.031536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:07.031536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 26 14:31:07.031536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:07.031536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:07.031536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 26 14:31:07.031536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 26 14:31:07.031536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 26 14:31:07.031536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 26 14:31:07.032508 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 14:31:07.032508 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application next(self.gen) Jul 26 14:31:07.032508 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 26 14:31:07.032508 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application yield resource Jul 26 14:31:07.032508 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 14:31:07.032508 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application next(self.gen) Jul 26 14:31:07.032508 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 26 14:31:07.032508 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self.session.rollback() Jul 26 14:31:07.032508 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 26 14:31:07.032508 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self.force_reraise() Jul 26 14:31:07.032508 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 26 14:31:07.032508 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise self.value Jul 26 14:31:07.032508 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 26 14:31:07.032508 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 26 14:31:07.032508 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 26 14:31:07.032508 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application session.commit() Jul 26 14:31:07.032508 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 26 14:31:07.033398 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 26 14:31:07.033398 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 26 14:31:07.033398 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._prepare_impl() Jul 26 14:31:07.033398 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 26 14:31:07.033398 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self.session.flush() Jul 26 14:31:07.033398 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 26 14:31:07.033398 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._flush(objects) Jul 26 14:31:07.033398 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 26 14:31:07.033398 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 26 14:31:07.033398 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 26 14:31:07.033398 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application compat.raise_( Jul 26 14:31:07.033398 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 14:31:07.033398 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception Jul 26 14:31:07.033398 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 26 14:31:07.033398 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application flush_context.execute() Jul 26 14:31:07.033398 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 26 14:31:07.033398 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rec.execute(self) Jul 26 14:31:07.034284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 26 14:31:07.034284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 26 14:31:07.034284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 26 14:31:07.034284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 26 14:31:07.034284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 26 14:31:07.034284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 26 14:31:07.034284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 26 14:31:07.034284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 26 14:31:07.034284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 26 14:31:07.034284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 26 14:31:07.034284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 26 14:31:07.034284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 26 14:31:07.034284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 26 14:31:07.034284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 26 14:31:07.034284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 26 14:31:07.035092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 26 14:31:07.035092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 14:31:07.035092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception Jul 26 14:31:07.035092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 14:31:07.035092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 14:31:07.035092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 14:31:07.035092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 14:31:07.035092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 14:31:07.035092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 14:31:07.035092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 14:31:07.035092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application conn.query(q) Jul 26 14:31:07.035092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 14:31:07.035092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 14:31:07.035092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 14:31:07.035092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application result.read() Jul 26 14:31:07.035092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 14:31:07.035092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 14:31:07.037660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 14:31:07.037660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 14:31:07.037660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 14:31:07.037660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 14:31:07.037660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 14:31:07.037660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 14:31:07.037660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 26 14:31:07.037660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 26 14:31:07.037660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [parameters: {'id': '0411764ec366422aa5e65f9663c2630b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 26 14:31:07.037660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 26 14:31:07.037660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:07.037660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:31:07.037660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:07.037660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:07.037660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 26 14:31:07.039639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 26 14:31:07.039639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 26 14:31:07.039639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 26 14:31:07.039639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 26 14:31:07.039639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:07.039639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:31:07.039639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:07.039639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:07.039639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:07.039639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:07.039639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:07.039639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:07.039639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:07.039639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:07.039639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:07.039639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:07.039639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:07.041778 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:07.041778 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 26 14:31:07.041778 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 26 14:31:07.041778 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:07.041778 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:07.041778 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 26 14:31:07.041778 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 26 14:31:07.041778 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0411764ec366422aa5e65f9663c2630b. Jul 26 14:31:07.041778 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:07.049688 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 466/937] 10.222.0.97 () {60 vars in 1140 bytes} [Tue Jul 26 14:31:06 2022] POST /identity/v3/projects => generated 262 bytes in 294 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:31:07.051157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40edeba2-9ef7-4bc8-96b6-b2c2a0ab1e91 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:07.085172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b8a63-e87d-424d-96a6-fe85a9b6b42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:07.085832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b8a63-e87d-424d-96a6-fe85a9b6b42a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:07.086364 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b8a63-e87d-424d-96a6-fe85a9b6b42a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:07.086491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b8a63-e87d-424d-96a6-fe85a9b6b42a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:07.094901 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b8a63-e87d-424d-96a6-fe85a9b6b42a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:07.096326 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b8a63-e87d-424d-96a6-fe85a9b6b42a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:07.109570 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-8d8b8a63-e87d-424d-96a6-fe85a9b6b42a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:31:07.109570 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:07.109570 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:07.109570 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:07.109570 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:07.109570 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:07.109570 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:07.109570 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:07.109570 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:07.109570 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:07.109570 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:07.109570 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:07.109570 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:31:07.109570 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:31:07.109570 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:31:07.110486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:31:07.110486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:07.110486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:07.110486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:07.110486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:07.110486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:07.110486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:07.110486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:07.110486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:07.110486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:07.110486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:07.110486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:07.110486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:07.110486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:07.110486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:07.110486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:07.110486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:07.111343 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:31:07.111343 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:31:07.111343 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:31:07.111343 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:31:07.111343 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:31:07.111343 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:31:07.111343 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:31:07.111343 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:07.111343 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 472/938] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:31:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:31:07.117758 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b8a63-e87d-424d-96a6-fe85a9b6b42a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:07.170998 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40edeba2-9ef7-4bc8-96b6-b2c2a0ab1e91 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:07.191462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:07.211261 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46f300-c1ac-4ca4-b451-daabfc8da55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:07.212327 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46f300-c1ac-4ca4-b451-daabfc8da55c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:07.212807 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46f300-c1ac-4ca4-b451-daabfc8da55c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:07.213400 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46f300-c1ac-4ca4-b451-daabfc8da55c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:07.221350 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46f300-c1ac-4ca4-b451-daabfc8da55c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:07.224097 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46f300-c1ac-4ca4-b451-daabfc8da55c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:07.226203 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b8a63-e87d-424d-96a6-fe85a9b6b42a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:07.236004 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-db46f300-c1ac-4ca4-b451-daabfc8da55c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:31:07.236004 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:07.236004 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:07.236004 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:07.236004 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:07.236004 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:07.236004 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:07.236004 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:07.236004 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:07.236004 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:07.236004 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:07.236004 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:07.236004 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:31:07.236004 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:31:07.236004 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:31:07.237252 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:31:07.237252 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:07.237252 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:07.237252 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:07.237252 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:07.237252 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:07.237252 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:07.237252 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:07.237252 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:07.237252 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:07.237252 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:07.237252 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:07.237252 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:07.237252 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:07.237252 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:07.237252 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:07.237252 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:07.238173 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:31:07.238173 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:31:07.238173 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:31:07.238173 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:31:07.238173 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:31:07.238173 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:31:07.238173 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:31:07.238173 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:07.242465 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 467/939] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:31:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:31:07.243623 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46f300-c1ac-4ca4-b451-daabfc8da55c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:07.258441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:07.264793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5f206-bc0a-4518-87c0-2ffdc73b3133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:07.265811 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5f206-bc0a-4518-87c0-2ffdc73b3133 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:07.266283 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5f206-bc0a-4518-87c0-2ffdc73b3133 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:07.266640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5f206-bc0a-4518-87c0-2ffdc73b3133 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:07.267433 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b5f206-bc0a-4518-87c0-2ffdc73b3133 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:07.267786 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b5f206-bc0a-4518-87c0-2ffdc73b3133 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:07.269388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b5f206-bc0a-4518-87c0-2ffdc73b3133 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:07.290163 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 473/940] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:31:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:07.291699 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5f206-bc0a-4518-87c0-2ffdc73b3133 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:07.358220 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46f300-c1ac-4ca4-b451-daabfc8da55c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:07.400421 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f922b-aa9a-49b4-8ebd-129bc4710996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:07.401055 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f922b-aa9a-49b4-8ebd-129bc4710996 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:07.401338 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f922b-aa9a-49b4-8ebd-129bc4710996 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:07.401617 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f922b-aa9a-49b4-8ebd-129bc4710996 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:07.402343 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f922b-aa9a-49b4-8ebd-129bc4710996 None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:07.402655 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f922b-aa9a-49b4-8ebd-129bc4710996 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:07.404143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f922b-aa9a-49b4-8ebd-129bc4710996 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:07.421551 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5f206-bc0a-4518-87c0-2ffdc73b3133 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:07.433231 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 468/941] 10.222.0.97 () {56 vars in 1112 bytes} [Tue Jul 26 14:31:07 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:07.433231 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc968c9f-0c51-496f-841a-7e94661b5815 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:07.433231 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc968c9f-0c51-496f-841a-7e94661b5815 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:07.433231 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc968c9f-0c51-496f-841a-7e94661b5815 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:07.435564 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 469/942] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:07.437058 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c525adb-20a5-4f06-994d-72d0e7190766 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:07.437346 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c525adb-20a5-4f06-994d-72d0e7190766 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:07.437624 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c525adb-20a5-4f06-994d-72d0e7190766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:07.465714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-0c525adb-20a5-4f06-994d-72d0e7190766 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:07.467384 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0c525adb-20a5-4f06-994d-72d0e7190766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:07.497077 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78f9ee-cbf7-42e6-9a5e-235f6a7d2162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:07.497779 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78f9ee-cbf7-42e6-9a5e-235f6a7d2162 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:07.498063 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78f9ee-cbf7-42e6-9a5e-235f6a7d2162 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:07.498335 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78f9ee-cbf7-42e6-9a5e-235f6a7d2162 None admin] PATH_INFO: `/v3/projects/c4c8da3011294783b2180ade1bfd9366/users/5179c3c4eaaa41f3beb79ba7f1993478/roles/a61134ee45dd49b3a20a8a55044e0e3c` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:07.503087 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a78f9ee-cbf7-42e6-9a5e-235f6a7d2162 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4c8da3011294783b2180ade1bfd9366, user_id=5179c3c4eaaa41f3beb79ba7f1993478, role_id=a61134ee45dd49b3a20a8a55044e0e3c)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:07.504630 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a78f9ee-cbf7-42e6-9a5e-235f6a7d2162 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:07.512859 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:07.559000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 470/943] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:07.587128 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c525adb-20a5-4f06-994d-72d0e7190766 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:07.669019 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 474/944] 10.222.0.97 () {58 vars in 1439 bytes} [Tue Jul 26 14:31:07 2022] PUT /identity/v3/projects/c4c8da3011294783b2180ade1bfd9366/users/5179c3c4eaaa41f3beb79ba7f1993478/roles/a61134ee45dd49b3a20a8a55044e0e3c => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:31:07.739099 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c525adb-20a5-4f06-994d-72d0e7190766 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:07.788413 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e323545-efd6-47f9-9e37-a31bc460cd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:07.789104 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e323545-efd6-47f9-9e37-a31bc460cd97 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:07.789390 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e323545-efd6-47f9-9e37-a31bc460cd97 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:07.790170 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e323545-efd6-47f9-9e37-a31bc460cd97 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:07.790827 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e323545-efd6-47f9-9e37-a31bc460cd97 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:07.792287 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e323545-efd6-47f9-9e37-a31bc460cd97 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:07.805050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-8e323545-efd6-47f9-9e37-a31bc460cd97 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 14:31:07.805050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:07.805050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:31:07.805050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:31:07.805050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:31:07.805050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:31:07.805050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:31:07.805050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:31:07.805050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 14:31:07.805050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:07.805050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:31:07.805050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:07.805050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:07.805050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:07.805050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:07.805050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:07.805050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:07.806119 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:07.806119 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:07.806119 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:07.806119 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:07.806119 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:07.806119 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:07.806119 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:31:07.806119 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:31:07.806119 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:31:07.806119 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:31:07.806119 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:07.806119 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:07.806119 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:07.806119 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:07.806119 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:07.806119 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:07.806939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:07.806939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:07.806939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:07.806939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:07.806939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:07.806939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:07.806939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:07.806939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:07.806939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:07.806939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:07.806939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:31:07.806939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:31:07.806939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 14:31:07.806939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:07.810714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 471/945] 10.222.0.97 () {56 vars in 1126 bytes} [Tue Jul 26 14:31:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:31:07.814775 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78f9ee-cbf7-42e6-9a5e-235f6a7d2162 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:07.890757 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:07.959722 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78f9ee-cbf7-42e6-9a5e-235f6a7d2162 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:08.011421 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6273ec4-6696-4beb-a379-ebe90756d886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:08.012548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6273ec4-6696-4beb-a379-ebe90756d886 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:08.013116 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6273ec4-6696-4beb-a379-ebe90756d886 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:08.013665 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6273ec4-6696-4beb-a379-ebe90756d886 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:08.015041 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6273ec4-6696-4beb-a379-ebe90756d886 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:08.015671 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6273ec4-6696-4beb-a379-ebe90756d886 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:08.017910 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6273ec4-6696-4beb-a379-ebe90756d886 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:08.034683 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 475/946] 10.222.0.97 () {56 vars in 1122 bytes} [Tue Jul 26 14:31:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:08.037216 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e323545-efd6-47f9-9e37-a31bc460cd97 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:08.094870 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:08.200697 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e323545-efd6-47f9-9e37-a31bc460cd97 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:08.262011 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f4a12-7892-4be3-9370-5b04eb3226f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:08.262942 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f4a12-7892-4be3-9370-5b04eb3226f4 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:08.263446 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f4a12-7892-4be3-9370-5b04eb3226f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:08.264110 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f4a12-7892-4be3-9370-5b04eb3226f4 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:08.265252 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f4a12-7892-4be3-9370-5b04eb3226f4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:08.265741 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f4a12-7892-4be3-9370-5b04eb3226f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:08.268221 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f4a12-7892-4be3-9370-5b04eb3226f4 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:08.278800 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 472/947] 10.222.0.97 () {56 vars in 1122 bytes} [Tue Jul 26 14:31:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:08.288406 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6273ec4-6696-4beb-a379-ebe90756d886 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:08.500265 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6273ec4-6696-4beb-a379-ebe90756d886 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:08.525358 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:08.559516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7bfb62-3ea7-40ae-bd78-2361c67aedda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:08.560435 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bfb62-3ea7-40ae-bd78-2361c67aedda None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:08.560944 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bfb62-3ea7-40ae-bd78-2361c67aedda None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:08.561399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bfb62-3ea7-40ae-bd78-2361c67aedda None admin] PATH_INFO: `/v3/endpoints` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:08.562694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7bfb62-3ea7-40ae-bd78-2361c67aedda None admin] RBAC: Adding query filter params (interface=public, service_id=5e2fa717d1d64eaf98a52757a50ea6b9, region_id=RegionOne) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:08.563253 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7bfb62-3ea7-40ae-bd78-2361c67aedda None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:08.566168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7bfb62-3ea7-40ae-bd78-2361c67aedda None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:08.577291 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 476/948] 10.222.0.97 () {56 vars in 1251 bytes} [Tue Jul 26 14:31:08 2022] GET /identity/v3/endpoints?service_id=5e2fa717d1d64eaf98a52757a50ea6b9&interface=public®ion_id=RegionOne => generated 194 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:08.584045 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f4a12-7892-4be3-9370-5b04eb3226f4 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:08.727896 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:08.736201 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f4a12-7892-4be3-9370-5b04eb3226f4 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:08.782260 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87861cab-5e7d-4496-875a-6613919a83c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:08.782872 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87861cab-5e7d-4496-875a-6613919a83c8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:08.783131 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87861cab-5e7d-4496-875a-6613919a83c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:08.783427 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87861cab-5e7d-4496-875a-6613919a83c8 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:08.784353 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87861cab-5e7d-4496-875a-6613919a83c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:08.785976 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87861cab-5e7d-4496-875a-6613919a83c8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:08.795802 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 473/949] 10.222.0.97 () {56 vars in 1087 bytes} [Tue Jul 26 14:31:08 2022] GET /identity/v3/services => generated 1497 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:31:08.878560 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae20ea-141c-4eff-b55a-edcf4fa4daf2 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:08.878917 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae20ea-141c-4eff-b55a-edcf4fa4daf2 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:08.879108 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae20ea-141c-4eff-b55a-edcf4fa4daf2 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:08.880943 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 477/950] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:08.886833 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07d8dc-267e-4994-a03e-aa45b326c164 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:08.887148 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07d8dc-267e-4994-a03e-aa45b326c164 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:08.887386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07d8dc-267e-4994-a03e-aa45b326c164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:08.908757 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-fb07d8dc-267e-4994-a03e-aa45b326c164 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:08.909997 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-fb07d8dc-267e-4994-a03e-aa45b326c164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:09.004919 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 474/951] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:09.031781 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae20ea-141c-4eff-b55a-edcf4fa4daf2 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:09.187675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae20ea-141c-4eff-b55a-edcf4fa4daf2 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:09.231266 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13232f0a-af7f-4e14-befd-f68f4b374a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:09.231949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13232f0a-af7f-4e14-befd-f68f4b374a56 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:09.232339 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13232f0a-af7f-4e14-befd-f68f4b374a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:09.232612 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13232f0a-af7f-4e14-befd-f68f4b374a56 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:09.233241 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13232f0a-af7f-4e14-befd-f68f4b374a56 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:09.234589 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13232f0a-af7f-4e14-befd-f68f4b374a56 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:09.245183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-13232f0a-af7f-4e14-befd-f68f4b374a56 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 14:31:09.245183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:09.245183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:31:09.245183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:31:09.245183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:31:09.245183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:31:09.245183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:31:09.245183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:31:09.245183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 14:31:09.245183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:09.245183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:31:09.245183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:09.245183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:09.245183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:09.245183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:09.245183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:09.245183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:09.246326 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:09.246326 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:09.246326 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:09.246326 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:09.246326 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:09.246326 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:09.246326 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:31:09.246326 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:31:09.246326 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:31:09.246326 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:31:09.246326 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:09.246326 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:09.246326 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:09.246326 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:09.246326 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:09.246326 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:09.247879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:09.247879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:09.247879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:09.247879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:09.247879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:09.247879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:09.247879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:09.247879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:09.247879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:09.247879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:09.247879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:31:09.247879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:31:09.247879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 14:31:09.247879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:09.247879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 478/952] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:31:09 2022] GET /identity/v3/services/compute => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:31:09.252585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb07d8dc-267e-4994-a03e-aa45b326c164 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:09.391389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:09.405478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb07d8dc-267e-4994-a03e-aa45b326c164 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:09.444531 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9bde1b-e3da-4279-a0ff-c9e65126e068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:09.445151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bde1b-e3da-4279-a0ff-c9e65126e068 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:09.445429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bde1b-e3da-4279-a0ff-c9e65126e068 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:09.445701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bde1b-e3da-4279-a0ff-c9e65126e068 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:09.446376 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9bde1b-e3da-4279-a0ff-c9e65126e068 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:09.446676 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9bde1b-e3da-4279-a0ff-c9e65126e068 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:09.448311 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9bde1b-e3da-4279-a0ff-c9e65126e068 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:09.453573 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 475/953] 10.222.0.97 () {56 vars in 1112 bytes} [Tue Jul 26 14:31:09 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:09.460744 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13232f0a-af7f-4e14-befd-f68f4b374a56 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:09.505807 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:09.576097 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13232f0a-af7f-4e14-befd-f68f4b374a56 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:09.617213 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05c8a6-dc37-4d3d-b2a1-0c013c7e11a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:09.617829 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05c8a6-dc37-4d3d-b2a1-0c013c7e11a2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:09.618109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05c8a6-dc37-4d3d-b2a1-0c013c7e11a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:09.618392 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05c8a6-dc37-4d3d-b2a1-0c013c7e11a2 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:09.619066 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05c8a6-dc37-4d3d-b2a1-0c013c7e11a2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:09.619366 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05c8a6-dc37-4d3d-b2a1-0c013c7e11a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:09.620746 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05c8a6-dc37-4d3d-b2a1-0c013c7e11a2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:09.625955 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 479/954] 10.222.0.97 () {56 vars in 1112 bytes} [Tue Jul 26 14:31:09 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:10.163726 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0a747-f82a-4a93-87db-971642601ca7 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:10.164271 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0a747-f82a-4a93-87db-971642601ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:10.164271 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0a747-f82a-4a93-87db-971642601ca7 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:10.165938 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 476/955] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:10.171390 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7c6a7-5b93-4655-b3d2-0f960fde5ec9 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:10.171863 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7c6a7-5b93-4655-b3d2-0f960fde5ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:10.171965 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7c6a7-5b93-4655-b3d2-0f960fde5ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:10.191321 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-7bc7c6a7-5b93-4655-b3d2-0f960fde5ec9 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:10.193141 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7bc7c6a7-5b93-4655-b3d2-0f960fde5ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:10.212889 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872e6d8-8bdb-4cbb-bf38-610c5d5d7550 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:10.213197 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872e6d8-8bdb-4cbb-bf38-610c5d5d7550 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:10.213429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872e6d8-8bdb-4cbb-bf38-610c5d5d7550 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:10.215136 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 477/956] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:10.220565 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094530d-90a6-428e-bf36-572f6a68862a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:10.220908 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094530d-90a6-428e-bf36-572f6a68862a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:10.221184 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094530d-90a6-428e-bf36-572f6a68862a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:10.245956 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-4094530d-90a6-428e-bf36-572f6a68862a None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:10.247161 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4094530d-90a6-428e-bf36-572f6a68862a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:10.278182 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 480/957] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:10.301547 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc7c6a7-5b93-4655-b3d2-0f960fde5ec9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:10.338696 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 478/958] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:10.355770 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:10.365159 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4094530d-90a6-428e-bf36-572f6a68862a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:10.417397 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc7c6a7-5b93-4655-b3d2-0f960fde5ec9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:10.459253 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6928162-f022-4fcb-b70a-00394591eb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:10.460028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6928162-f022-4fcb-b70a-00394591eb2e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:10.460386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6928162-f022-4fcb-b70a-00394591eb2e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:10.460731 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6928162-f022-4fcb-b70a-00394591eb2e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:10.472015 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:10.472993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6928162-f022-4fcb-b70a-00394591eb2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:31:10.481079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-b6928162-f022-4fcb-b70a-00394591eb2e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 14:31:10.481079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:10.481079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:10.481079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:10.481079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:10.481079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:10.481079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:10.481079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:10.481079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:10.481079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:10.481079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:10.481079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:10.481079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:31:10.481079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:31:10.481079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:31:10.481079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:31:10.489023 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:31:10.489023 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:31:10.489023 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:31:10.489023 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:31:10.489023 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:10.489023 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:10.489023 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:10.489023 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:10.489023 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:10.489023 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:10.489023 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:10.489023 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:10.489023 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:10.489023 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:10.489023 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:10.489023 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:10.489917 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:10.489917 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:10.489917 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:10.489917 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:10.489917 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:31:10.489917 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:31:10.489917 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:31:10.489917 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:31:10.489917 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:31:10.489917 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:31:10.489917 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 14:31:10.489917 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:10.491087 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 481/959] 10.222.0.97 () {56 vars in 1120 bytes} [Tue Jul 26 14:31:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:31:10.491482 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4094530d-90a6-428e-bf36-572f6a68862a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:10.492405 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6928162-f022-4fcb-b70a-00394591eb2e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:10.540052 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cfcba3-29e4-4526-81f2-3459ed1469c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:10.541246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfcba3-29e4-4526-81f2-3459ed1469c7 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:10.541528 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfcba3-29e4-4526-81f2-3459ed1469c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:10.541798 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfcba3-29e4-4526-81f2-3459ed1469c7 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:10.542489 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cfcba3-29e4-4526-81f2-3459ed1469c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:10.544397 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cfcba3-29e4-4526-81f2-3459ed1469c7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:10.602455 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:10.604870 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.sql.core [None req-d9cfcba3-29e4-4526-81f2-3459ed1469c7 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 26 14:31:10.604870 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 26 14:31:10.604870 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [parameters: {'user_id': 'f63d7ea3764649aebde520b6e1df7625', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 26 14:31:10.604870 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80523) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 26 14:31:10.609516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6928162-f022-4fcb-b70a-00394591eb2e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:10.614577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-d9cfcba3-29e4-4526-81f2-3459ed1469c7 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 26 14:31:10.614577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:10.614577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 14:31:10.614577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 14:31:10.614577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 14:31:10.614577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 14:31:10.614577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 14:31:10.614577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 14:31:10.614577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 14:31:10.614577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application conn.query(q) Jul 26 14:31:10.614577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 14:31:10.614577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 14:31:10.614577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 14:31:10.614577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application result.read() Jul 26 14:31:10.614577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 14:31:10.621545 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 14:31:10.621545 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 14:31:10.621545 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 14:31:10.621545 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 14:31:10.621545 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 14:31:10.621545 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 14:31:10.621545 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 14:31:10.621545 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 26 14:31:10.621545 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:10.621545 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 26 14:31:10.621545 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:10.621545 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:10.621545 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 26 14:31:10.621545 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 26 14:31:10.621545 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 26 14:31:10.621545 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 26 14:31:10.621545 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 14:31:10.621545 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application next(self.gen) Jul 26 14:31:10.623101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 26 14:31:10.623101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application yield resource Jul 26 14:31:10.623101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 14:31:10.623101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application next(self.gen) Jul 26 14:31:10.623101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 26 14:31:10.623101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self.session.rollback() Jul 26 14:31:10.623101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 26 14:31:10.623101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self.force_reraise() Jul 26 14:31:10.623101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 26 14:31:10.623101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise self.value Jul 26 14:31:10.623101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 26 14:31:10.623101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 26 14:31:10.623101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 26 14:31:10.623101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application session.commit() Jul 26 14:31:10.623101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 26 14:31:10.623101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 26 14:31:10.623101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 26 14:31:10.624928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._prepare_impl() Jul 26 14:31:10.624928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 26 14:31:10.624928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self.session.flush() Jul 26 14:31:10.624928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 26 14:31:10.624928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._flush(objects) Jul 26 14:31:10.624928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 26 14:31:10.624928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 26 14:31:10.624928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 26 14:31:10.624928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application compat.raise_( Jul 26 14:31:10.624928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 14:31:10.624928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception Jul 26 14:31:10.624928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 26 14:31:10.624928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application flush_context.execute() Jul 26 14:31:10.624928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 26 14:31:10.624928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rec.execute(self) Jul 26 14:31:10.624928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 26 14:31:10.624928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 26 14:31:10.626358 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 26 14:31:10.626358 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 26 14:31:10.626358 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 26 14:31:10.626358 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 26 14:31:10.626358 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 26 14:31:10.626358 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 26 14:31:10.626358 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 26 14:31:10.626358 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 26 14:31:10.626358 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 26 14:31:10.626358 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 26 14:31:10.626358 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 26 14:31:10.626358 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 26 14:31:10.626358 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 26 14:31:10.626358 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 26 14:31:10.626358 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 14:31:10.626358 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception Jul 26 14:31:10.628074 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 14:31:10.628074 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 14:31:10.628074 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 14:31:10.628074 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 14:31:10.628074 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 14:31:10.628074 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 14:31:10.628074 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 14:31:10.628074 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application conn.query(q) Jul 26 14:31:10.628074 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 14:31:10.628074 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 14:31:10.628074 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 14:31:10.628074 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application result.read() Jul 26 14:31:10.628074 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 14:31:10.628074 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 14:31:10.628074 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 14:31:10.628074 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 14:31:10.629448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 14:31:10.629448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 14:31:10.629448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 14:31:10.629448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 14:31:10.629448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 26 14:31:10.629448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 26 14:31:10.629448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [parameters: {'user_id': 'f63d7ea3764649aebde520b6e1df7625', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 26 14:31:10.629448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 26 14:31:10.629448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:10.629448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:31:10.629448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:10.629448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:10.629448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:10.629448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:10.629448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:10.631185 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:10.631185 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:10.631185 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:10.631185 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:10.631185 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:10.631185 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:10.631185 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:10.631185 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 26 14:31:10.631185 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 26 14:31:10.631185 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:10.631185 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:10.631185 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:31:10.631185 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:31:10.631185 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:31:10.631185 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:31:10.631185 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 26 14:31:10.631185 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 26 14:31:10.636820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 26 14:31:10.636820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 26 14:31:10.636820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 26 14:31:10.636820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 26 14:31:10.636820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 26 14:31:10.636820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:10.636820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 479/960] 10.222.0.97 () {60 vars in 1131 bytes} [Tue Jul 26 14:31:10 2022] POST /identity/v3/users => generated 160 bytes in 252 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:31:10.636820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cfcba3-29e4-4526-81f2-3459ed1469c7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:10.678503 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c7c4a1-00bc-4dbc-9a09-f5766e564c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:10.679868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7c4a1-00bc-4dbc-9a09-f5766e564c87 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:10.680647 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7c4a1-00bc-4dbc-9a09-f5766e564c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:10.681280 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7c4a1-00bc-4dbc-9a09-f5766e564c87 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:10.682644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c7c4a1-00bc-4dbc-9a09-f5766e564c87 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:10.683347 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c7c4a1-00bc-4dbc-9a09-f5766e564c87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:10.686073 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c7c4a1-00bc-4dbc-9a09-f5766e564c87 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:10.702805 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 482/961] 10.222.0.97 () {56 vars in 1115 bytes} [Tue Jul 26 14:31:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:10.704099 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c7c4a1-00bc-4dbc-9a09-f5766e564c87 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:10.770707 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cfcba3-29e4-4526-81f2-3459ed1469c7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:10.814925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a84f08-882f-41f3-a6b4-4e34ec6fdd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:10.815955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a84f08-882f-41f3-a6b4-4e34ec6fdd97 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:10.816422 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a84f08-882f-41f3-a6b4-4e34ec6fdd97 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:10.816894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a84f08-882f-41f3-a6b4-4e34ec6fdd97 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:10.825646 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a84f08-882f-41f3-a6b4-4e34ec6fdd97 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:10.827452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a84f08-882f-41f3-a6b4-4e34ec6fdd97 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:10.841748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-65a84f08-882f-41f3-a6b4-4e34ec6fdd97 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 14:31:10.841748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:10.841748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:10.841748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:10.841748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:10.841748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:10.841748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:10.841748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:10.841748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:10.841748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:10.841748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:10.841748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:10.841748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:31:10.841748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:31:10.841748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:31:10.841748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:31:10.842935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:10.842935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:10.842935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:31:10.842935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:31:10.842935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:31:10.842935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:31:10.842935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:10.842935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:10.842935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:10.842935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:10.842935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:10.842935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:10.842935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:10.842935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:10.842935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:10.842935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:10.842935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:10.842935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:10.844138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:10.844138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:10.844138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:31:10.844138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:31:10.844138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:31:10.844138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:31:10.844138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:31:10.844138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:31:10.844138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 14:31:10.844138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:10.850220 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 480/962] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:31:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:31:10.851531 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a84f08-882f-41f3-a6b4-4e34ec6fdd97 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:10.862575 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:10.870953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c7c4a1-00bc-4dbc-9a09-f5766e564c87 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:10.952978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c9f9a-63b5-416c-97e3-6bd0bfe825ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:10.954434 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c9f9a-63b5-416c-97e3-6bd0bfe825ac None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:10.955290 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c9f9a-63b5-416c-97e3-6bd0bfe825ac None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:10.955821 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c9f9a-63b5-416c-97e3-6bd0bfe825ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:10.980028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a84f08-882f-41f3-a6b4-4e34ec6fdd97 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:10.984132 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c9f9a-63b5-416c-97e3-6bd0bfe825ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:10.991511 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c9f9a-63b5-416c-97e3-6bd0bfe825ac None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:11.019490 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c420c-24b3-4016-a2ac-5563a1bdc786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:11.020302 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c420c-24b3-4016-a2ac-5563a1bdc786 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:11.020694 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c420c-24b3-4016-a2ac-5563a1bdc786 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:11.021071 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c420c-24b3-4016-a2ac-5563a1bdc786 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:11.021936 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1c420c-24b3-4016-a2ac-5563a1bdc786 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:11.022087 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-8d5c9f9a-63b5-416c-97e3-6bd0bfe825ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:31:11.022087 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:11.022087 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:11.022087 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:11.022087 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:11.022087 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:11.022087 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:11.022087 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:11.022087 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:11.022087 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:11.022087 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:11.022087 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:11.022087 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:31:11.022087 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:31:11.022087 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:31:11.023109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:31:11.023109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:11.023109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:11.023109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:11.023109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:11.023109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:11.023109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:11.023109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:11.023109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:11.023109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:11.023109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:11.023109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:11.023109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:11.023109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:11.023109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:11.023109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:11.023109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:11.024257 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:31:11.024257 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:31:11.024257 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:31:11.024257 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:11.024495 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1c420c-24b3-4016-a2ac-5563a1bdc786 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:11.024654 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 483/963] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:31:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:31:11.026745 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6b529-0d3d-4a2e-a675-207c5c03bda2 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:11.027246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6b529-0d3d-4a2e-a675-207c5c03bda2 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:11.030267 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6b529-0d3d-4a2e-a675-207c5c03bda2 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:11.030428 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1c420c-24b3-4016-a2ac-5563a1bdc786 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:11.030612 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 484/964] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:11.031456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6b529-0d3d-4a2e-a675-207c5c03bda2 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:11.057381 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:11.079797 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 481/965] 10.222.0.97 () {56 vars in 1101 bytes} [Tue Jul 26 14:31:10 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:11.083286 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25304df2-db43-4937-bfcc-3884ba9fa3df None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:11.084751 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25304df2-db43-4937-bfcc-3884ba9fa3df None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:11.085205 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25304df2-db43-4937-bfcc-3884ba9fa3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:11.111552 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-25304df2-db43-4937-bfcc-3884ba9fa3df None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:11.114350 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-25304df2-db43-4937-bfcc-3884ba9fa3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:11.176418 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6b529-0d3d-4a2e-a675-207c5c03bda2 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:11.207363 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:11.221409 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 482/966] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:11.235426 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518d2a5-53c6-46e1-87b1-ca32290bb774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:11.235426 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518d2a5-53c6-46e1-87b1-ca32290bb774 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:11.235426 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518d2a5-53c6-46e1-87b1-ca32290bb774 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:11.235426 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518d2a5-53c6-46e1-87b1-ca32290bb774 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:11.235891 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518d2a5-53c6-46e1-87b1-ca32290bb774 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:11.235891 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518d2a5-53c6-46e1-87b1-ca32290bb774 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:11.235891 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518d2a5-53c6-46e1-87b1-ca32290bb774 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:11.249724 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 485/967] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:11.250382 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25304df2-db43-4937-bfcc-3884ba9fa3df None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:11.257419 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518d2a5-53c6-46e1-87b1-ca32290bb774 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:11.359999 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:11.397572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25304df2-db43-4937-bfcc-3884ba9fa3df None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:11.417800 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518d2a5-53c6-46e1-87b1-ca32290bb774 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:11.446396 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c2f70-77c2-4955-bca1-f1dfeb6bc972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:11.447080 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c2f70-77c2-4955-bca1-f1dfeb6bc972 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:11.447364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c2f70-77c2-4955-bca1-f1dfeb6bc972 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:11.447664 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c2f70-77c2-4955-bca1-f1dfeb6bc972 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:11.448297 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c2f70-77c2-4955-bca1-f1dfeb6bc972 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:11.449701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c2f70-77c2-4955-bca1-f1dfeb6bc972 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:11.460987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-1e2c2f70-77c2-4955-bca1-f1dfeb6bc972 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 14:31:11.460987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:11.460987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:31:11.460987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:31:11.460987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:31:11.460987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:31:11.460987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:31:11.460987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:31:11.460987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 14:31:11.460987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:11.460987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:31:11.460987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:11.460987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:11.460987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:11.460987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:11.460987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:11.460987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:11.463366 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:11.463366 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:11.463366 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:11.463366 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:11.463366 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:11.463366 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:11.463366 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:31:11.463366 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:31:11.463366 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:31:11.463366 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:31:11.463366 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:11.463366 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:11.463366 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:11.463366 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:11.463366 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:11.463366 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:11.466091 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:11.466091 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:11.466091 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:11.466091 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:11.466091 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:11.466091 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:11.466091 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:11.466091 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:11.466091 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:11.466091 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:11.466091 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:31:11.466091 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:31:11.466091 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 14:31:11.466091 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:11.466091 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 483/968] 10.222.0.97 () {56 vars in 1126 bytes} [Tue Jul 26 14:31:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:31:11.469904 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c2f70-77c2-4955-bca1-f1dfeb6bc972 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:11.473205 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c3f76-a90b-4dda-a8c1-5d404bce35f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:11.474009 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c3f76-a90b-4dda-a8c1-5d404bce35f4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:11.474295 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c3f76-a90b-4dda-a8c1-5d404bce35f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:11.474639 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c3f76-a90b-4dda-a8c1-5d404bce35f4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:11.484072 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c3f76-a90b-4dda-a8c1-5d404bce35f4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:11.485704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c3f76-a90b-4dda-a8c1-5d404bce35f4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:11.508363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-c38c3f76-a90b-4dda-a8c1-5d404bce35f4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 14:31:11.508363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:11.508363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:11.508363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:11.508363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:11.508363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:11.508363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:11.508363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:11.508363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:11.508363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:11.508363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:11.508363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:11.508363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:31:11.508363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:31:11.508363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:31:11.508363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:31:11.509600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:11.509600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:11.509600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:31:11.509600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:31:11.509600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:31:11.509600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:31:11.509600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:11.509600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:11.509600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:11.509600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:11.509600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:11.509600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:11.509600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:11.509600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:11.509600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:11.509600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:11.509600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:11.509600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:11.510868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:11.510868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:11.510868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:31:11.510868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:31:11.510868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:31:11.510868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:31:11.510868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:31:11.510868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:31:11.510868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 14:31:11.510868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:11.515109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 486/969] 10.222.0.97 () {56 vars in 1117 bytes} [Tue Jul 26 14:31:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:31:11.516264 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c3f76-a90b-4dda-a8c1-5d404bce35f4 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:11.542600 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:11.609333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c2f70-77c2-4955-bca1-f1dfeb6bc972 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:11.663067 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cdb4be-df35-4934-92ba-d73c18017247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:11.663986 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cdb4be-df35-4934-92ba-d73c18017247 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:11.664105 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c3f76-a90b-4dda-a8c1-5d404bce35f4 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:11.664448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cdb4be-df35-4934-92ba-d73c18017247 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:11.664883 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cdb4be-df35-4934-92ba-d73c18017247 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:11.666010 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cdb4be-df35-4934-92ba-d73c18017247 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:11.666516 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cdb4be-df35-4934-92ba-d73c18017247 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:11.668639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cdb4be-df35-4934-92ba-d73c18017247 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:11.677340 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 484/970] 10.222.0.97 () {56 vars in 1122 bytes} [Tue Jul 26 14:31:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:11.686127 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cdb4be-df35-4934-92ba-d73c18017247 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:11.714071 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:11.723512 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8323f67-d9d8-48a7-8f33-d87cf4fd7ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:11.724434 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8323f67-d9d8-48a7-8f33-d87cf4fd7ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:11.724657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8323f67-d9d8-48a7-8f33-d87cf4fd7ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:11.724936 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8323f67-d9d8-48a7-8f33-d87cf4fd7ef5 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:11.725747 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8323f67-d9d8-48a7-8f33-d87cf4fd7ef5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:11.726062 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8323f67-d9d8-48a7-8f33-d87cf4fd7ef5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:11.727752 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8323f67-d9d8-48a7-8f33-d87cf4fd7ef5 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:11.751436 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 487/971] 10.222.0.97 () {56 vars in 1149 bytes} [Tue Jul 26 14:31:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:11.761077 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8323f67-d9d8-48a7-8f33-d87cf4fd7ef5 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:11.820014 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cdb4be-df35-4934-92ba-d73c18017247 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:11.855105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:11.871034 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20267fe3-fe48-400b-8246-8f160fbf3d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:11.871854 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20267fe3-fe48-400b-8246-8f160fbf3d51 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:11.872140 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20267fe3-fe48-400b-8246-8f160fbf3d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:11.872415 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20267fe3-fe48-400b-8246-8f160fbf3d51 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:11.873112 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20267fe3-fe48-400b-8246-8f160fbf3d51 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:11.873416 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20267fe3-fe48-400b-8246-8f160fbf3d51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:11.874831 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20267fe3-fe48-400b-8246-8f160fbf3d51 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:11.880264 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 485/972] 10.222.0.97 () {56 vars in 1122 bytes} [Tue Jul 26 14:31:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:11.886582 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25764524-6f81-4bb1-975c-4ef38e063256 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:11.886870 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25764524-6f81-4bb1-975c-4ef38e063256 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:11.887600 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25764524-6f81-4bb1-975c-4ef38e063256 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:11.889134 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 486/973] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:11.889866 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25764524-6f81-4bb1-975c-4ef38e063256 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:11.898853 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8323f67-d9d8-48a7-8f33-d87cf4fd7ef5 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:11.951842 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b0a68-c746-4c69-802d-15ea8f84c5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:11.952476 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b0a68-c746-4c69-802d-15ea8f84c5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:11.952974 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b0a68-c746-4c69-802d-15ea8f84c5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:11.953467 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b0a68-c746-4c69-802d-15ea8f84c5d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:12.020365 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25764524-6f81-4bb1-975c-4ef38e063256 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:12.041635 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:12.054883 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b0a68-c746-4c69-802d-15ea8f84c5d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:12.056487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b0a68-c746-4c69-802d-15ea8f84c5d4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:12.061635 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1277a516-dea0-408f-a6ba-c75786478663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:12.062259 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1277a516-dea0-408f-a6ba-c75786478663 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:12.062544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1277a516-dea0-408f-a6ba-c75786478663 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:12.062821 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1277a516-dea0-408f-a6ba-c75786478663 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:12.063574 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1277a516-dea0-408f-a6ba-c75786478663 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:12.064931 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1277a516-dea0-408f-a6ba-c75786478663 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:12.147698 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 487/974] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:31:11 2022] POST /identity/v3/endpoints => generated 349 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:31:12.148956 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ffbc5-70f0-488c-9b71-1139e37c2c2d None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:12.149248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ffbc5-70f0-488c-9b71-1139e37c2c2d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:12.149525 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ffbc5-70f0-488c-9b71-1139e37c2c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:12.164447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 488/975] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:31:11 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 14:31:12.165678 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b0a68-c746-4c69-802d-15ea8f84c5d4 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:12.170245 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-4b8ffbc5-70f0-488c-9b71-1139e37c2c2d None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:12.171634 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4b8ffbc5-70f0-488c-9b71-1139e37c2c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:12.223167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:12.265690 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 488/976] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:12.280568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:12.292733 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ffbc5-70f0-488c-9b71-1139e37c2c2d None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:12.294156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b0a68-c746-4c69-802d-15ea8f84c5d4 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:12.332659 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e395c5f-2eda-4582-b3e0-0afe17215e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:12.334186 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e395c5f-2eda-4582-b3e0-0afe17215e73 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:12.334535 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e395c5f-2eda-4582-b3e0-0afe17215e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:12.334879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e395c5f-2eda-4582-b3e0-0afe17215e73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:12.349017 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e395c5f-2eda-4582-b3e0-0afe17215e73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:12.350612 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e395c5f-2eda-4582-b3e0-0afe17215e73 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:12.369179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-9e395c5f-2eda-4582-b3e0-0afe17215e73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:31:12.369179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:12.369179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:12.369179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:12.369179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:12.369179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:12.369179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:12.369179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:12.369179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:12.369179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:12.369179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:12.369179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:12.369179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:31:12.369179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:31:12.369179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:31:12.371657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:31:12.371657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:12.371657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:12.371657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:12.371657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:12.371657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:12.371657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:12.371657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:12.371657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:12.371657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:12.371657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:12.371657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:12.371657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:12.371657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:12.371657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:12.371657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:12.371657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:12.373491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:31:12.373491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:31:12.373491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:31:12.373491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:12.373491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 489/977] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:31:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:31:12.376303 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e395c5f-2eda-4582-b3e0-0afe17215e73 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:12.414085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ffbc5-70f0-488c-9b71-1139e37c2c2d None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:12.454715 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07baf2fc-6361-41b2-86ab-5fdc856aba5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:12.455348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07baf2fc-6361-41b2-86ab-5fdc856aba5f None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:12.455665 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07baf2fc-6361-41b2-86ab-5fdc856aba5f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:12.455940 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07baf2fc-6361-41b2-86ab-5fdc856aba5f None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:12.456669 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07baf2fc-6361-41b2-86ab-5fdc856aba5f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:12.458606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07baf2fc-6361-41b2-86ab-5fdc856aba5f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:12.489263 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e395c5f-2eda-4582-b3e0-0afe17215e73 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:12.525216 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b896a-8f68-43d8-87b0-fa58549666c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:12.525836 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b896a-8f68-43d8-87b0-fa58549666c9 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:12.526118 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b896a-8f68-43d8-87b0-fa58549666c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:12.526395 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b896a-8f68-43d8-87b0-fa58549666c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:12.527089 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b896a-8f68-43d8-87b0-fa58549666c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:12.527437 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b896a-8f68-43d8-87b0-fa58549666c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:12.528792 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b896a-8f68-43d8-87b0-fa58549666c9 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:12.538218 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:12.542025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 490/978] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:31:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:12.547727 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b896a-8f68-43d8-87b0-fa58549666c9 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:12.559569 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 489/979] 10.222.0.97 () {60 vars in 1141 bytes} [Tue Jul 26 14:31:12 2022] POST /identity/v3/services => generated 242 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:31:12.674392 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b896a-8f68-43d8-87b0-fa58549666c9 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:12.721781 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d02a5-43a1-49d7-95c9-f177e5872b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:12.722451 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d02a5-43a1-49d7-95c9-f177e5872b32 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:12.722737 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d02a5-43a1-49d7-95c9-f177e5872b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:12.723013 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d02a5-43a1-49d7-95c9-f177e5872b32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:12.730228 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d02a5-43a1-49d7-95c9-f177e5872b32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:12.731702 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d02a5-43a1-49d7-95c9-f177e5872b32 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:12.745879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-021d02a5-43a1-49d7-95c9-f177e5872b32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:31:12.745879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:12.745879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:12.745879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:12.745879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:12.745879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:12.745879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:12.745879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:12.745879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:12.745879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:12.745879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:12.745879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:12.745879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:31:12.745879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:31:12.745879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:31:12.747035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:31:12.747035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:12.747035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:12.747035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:12.747035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:12.747035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:12.747035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:12.747035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:12.747035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:12.747035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:12.747035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:12.747035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:12.747035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:12.747035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:12.747035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:12.747035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:12.747035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:12.748547 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:31:12.748547 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:31:12.748547 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:31:12.748547 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:31:12.748547 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:31:12.748547 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:31:12.748547 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:31:12.748547 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:12.748547 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 491/980] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:31:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:31:12.753876 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07baf2fc-6361-41b2-86ab-5fdc856aba5f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:12.785401 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:12.881828 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07baf2fc-6361-41b2-86ab-5fdc856aba5f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:12.923701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08102cec-b227-4cd3-b062-95daf7dc74ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:12.924663 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08102cec-b227-4cd3-b062-95daf7dc74ae None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:12.925094 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08102cec-b227-4cd3-b062-95daf7dc74ae None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:12.925516 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08102cec-b227-4cd3-b062-95daf7dc74ae None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:12.926499 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08102cec-b227-4cd3-b062-95daf7dc74ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:12.926961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08102cec-b227-4cd3-b062-95daf7dc74ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:12.928655 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08102cec-b227-4cd3-b062-95daf7dc74ae None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:12.943188 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 490/981] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:31:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:12.951419 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d02a5-43a1-49d7-95c9-f177e5872b32 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:13.014181 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:13.069750 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d02a5-43a1-49d7-95c9-f177e5872b32 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:13.105405 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f628a333-ca4d-4dbb-bc2a-74cb65c1c122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:13.106023 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628a333-ca4d-4dbb-bc2a-74cb65c1c122 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:13.106301 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628a333-ca4d-4dbb-bc2a-74cb65c1c122 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:13.106572 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628a333-ca4d-4dbb-bc2a-74cb65c1c122 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:13.107349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f628a333-ca4d-4dbb-bc2a-74cb65c1c122 None admin] RBAC: Adding query filter params (role.id=a61134ee45dd49b3a20a8a55044e0e3c, scope.project.id=c4c8da3011294783b2180ade1bfd9366, user.id=5179c3c4eaaa41f3beb79ba7f1993478) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:13.108312 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f628a333-ca4d-4dbb-bc2a-74cb65c1c122 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:13.109335 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f628a333-ca4d-4dbb-bc2a-74cb65c1c122 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:13.123486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 492/982] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:31:12 2022] GET /identity/v3/role_assignments?user.id=5179c3c4eaaa41f3beb79ba7f1993478&scope.project.id=c4c8da3011294783b2180ade1bfd9366&role.id=a61134ee45dd49b3a20a8a55044e0e3c => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:14.282562 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490a9ad-b670-4d9c-bcb2-a333a6e171d9 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:14.283104 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490a9ad-b670-4d9c-bcb2-a333a6e171d9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:14.283104 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490a9ad-b670-4d9c-bcb2-a333a6e171d9 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:14.284548 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 491/983] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:14.289751 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4820a-3dbf-4410-8aae-2f83cc55fea4 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:14.290171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4820a-3dbf-4410-8aae-2f83cc55fea4 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:14.290410 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4820a-3dbf-4410-8aae-2f83cc55fea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:14.310096 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-89c4820a-3dbf-4410-8aae-2f83cc55fea4 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:14.311212 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-89c4820a-3dbf-4410-8aae-2f83cc55fea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:14.383920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 493/984] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:14 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:14.406533 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490a9ad-b670-4d9c-bcb2-a333a6e171d9 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:14.498427 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:14.514867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490a9ad-b670-4d9c-bcb2-a333a6e171d9 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:14.554192 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f2fce-599b-4fd2-a944-c3ac011634e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:14.555231 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f2fce-599b-4fd2-a944-c3ac011634e5 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:14.555747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f2fce-599b-4fd2-a944-c3ac011634e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:14.556207 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f2fce-599b-4fd2-a944-c3ac011634e5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:14.557015 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f2fce-599b-4fd2-a944-c3ac011634e5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:14.558577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f2fce-599b-4fd2-a944-c3ac011634e5 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:14.569418 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-a68f2fce-599b-4fd2-a944-c3ac011634e5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 14:31:14.569418 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:14.569418 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:31:14.569418 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:31:14.569418 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:31:14.569418 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:31:14.569418 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:31:14.569418 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:31:14.569418 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 14:31:14.569418 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:14.569418 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:31:14.569418 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:14.569418 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:14.569418 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:14.569418 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:14.569418 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:14.569418 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:14.570555 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:14.570555 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:14.570555 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:14.570555 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:14.570555 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:14.570555 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:14.570555 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:31:14.570555 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:31:14.570555 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:31:14.570555 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:31:14.570555 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:14.570555 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:14.570555 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:14.570555 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:14.570555 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:14.570555 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:14.571710 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:14.571710 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:14.571710 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:14.571710 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:14.571710 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:14.571710 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:14.571710 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:14.571710 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:14.571710 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:14.571710 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:14.571710 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:31:14.571710 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:31:14.571710 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 14:31:14.571710 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:14.574070 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 492/985] 10.222.0.97 () {56 vars in 1126 bytes} [Tue Jul 26 14:31:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:31:14.580267 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4820a-3dbf-4410-8aae-2f83cc55fea4 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:14.635588 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:14.690244 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4820a-3dbf-4410-8aae-2f83cc55fea4 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:14.715951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43ba61-d7de-4a2a-a071-6bbf836141d2 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:14.716258 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43ba61-d7de-4a2a-a071-6bbf836141d2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:14.716533 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43ba61-d7de-4a2a-a071-6bbf836141d2 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:14.718050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 493/986] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:14.723553 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a405b-87ec-445e-8ecf-611500a28033 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:14.723835 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a405b-87ec-445e-8ecf-611500a28033 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:14.724120 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a405b-87ec-445e-8ecf-611500a28033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:14.727904 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892fa32f-8374-4063-bd6b-7226350dfe42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:14.728647 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fa32f-8374-4063-bd6b-7226350dfe42 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:14.729139 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fa32f-8374-4063-bd6b-7226350dfe42 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:14.729732 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fa32f-8374-4063-bd6b-7226350dfe42 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:14.731053 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892fa32f-8374-4063-bd6b-7226350dfe42 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:14.731495 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892fa32f-8374-4063-bd6b-7226350dfe42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:14.732969 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892fa32f-8374-4063-bd6b-7226350dfe42 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:14.738556 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 494/987] 10.222.0.97 () {56 vars in 1122 bytes} [Tue Jul 26 14:31:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:14.744453 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892fa32f-8374-4063-bd6b-7226350dfe42 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:14.748877 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-964a405b-87ec-445e-8ecf-611500a28033 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:14.750970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-964a405b-87ec-445e-8ecf-611500a28033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:14.835530 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 494/988] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:14 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 14:31:14.859204 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a405b-87ec-445e-8ecf-611500a28033 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:14.867526 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892fa32f-8374-4063-bd6b-7226350dfe42 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:14.882524 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:14.904057 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0dd1b1-b6b1-4a17-a336-d930429fdef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:14.904861 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0dd1b1-b6b1-4a17-a336-d930429fdef7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:14.905307 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0dd1b1-b6b1-4a17-a336-d930429fdef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:14.905761 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0dd1b1-b6b1-4a17-a336-d930429fdef7 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:14.907001 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0dd1b1-b6b1-4a17-a336-d930429fdef7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:14.907522 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0dd1b1-b6b1-4a17-a336-d930429fdef7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:14.911707 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0dd1b1-b6b1-4a17-a336-d930429fdef7 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:14.917511 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 495/989] 10.222.0.97 () {56 vars in 1122 bytes} [Tue Jul 26 14:31:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:14.922881 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0dd1b1-b6b1-4a17-a336-d930429fdef7 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:14.959344 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:14.979857 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a405b-87ec-445e-8ecf-611500a28033 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:15.023112 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05024f1-7b28-40a6-932f-34c212ba0eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:15.024154 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05024f1-7b28-40a6-932f-34c212ba0eca None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:15.024625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05024f1-7b28-40a6-932f-34c212ba0eca None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:15.025089 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05024f1-7b28-40a6-932f-34c212ba0eca None admin] PATH_INFO: `/v3/services/compute` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:15.026158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05024f1-7b28-40a6-932f-34c212ba0eca None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:15.028440 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05024f1-7b28-40a6-932f-34c212ba0eca None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:15.042900 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-c05024f1-7b28-40a6-932f-34c212ba0eca None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 14:31:15.042900 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:15.042900 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:31:15.042900 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:31:15.042900 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:31:15.042900 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:31:15.042900 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:31:15.042900 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:31:15.042900 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 14:31:15.042900 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:15.042900 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:31:15.042900 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:15.042900 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:15.042900 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:15.042900 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:15.042900 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:15.042900 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:15.044149 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:15.044149 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:15.044149 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:15.044149 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:15.044149 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:15.044149 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:15.044149 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:31:15.044149 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:31:15.044149 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:31:15.044149 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:31:15.044149 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:15.044149 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:15.044149 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:15.044149 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:15.044149 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:15.044149 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:15.047678 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:15.047678 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:15.047678 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:15.047678 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:15.047678 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:15.047678 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:15.047678 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:15.047678 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:15.047678 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:15.047678 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:15.047678 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:31:15.047678 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:31:15.047678 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 14:31:15.047678 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:15.047678 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 495/990] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:31:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:31:15.052008 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0dd1b1-b6b1-4a17-a336-d930429fdef7 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:15.054095 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05024f1-7b28-40a6-932f-34c212ba0eca None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:15.099371 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d3a2ef-2e67-4556-8009-4bc4f1bba932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:15.100425 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3a2ef-2e67-4556-8009-4bc4f1bba932 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:15.100719 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3a2ef-2e67-4556-8009-4bc4f1bba932 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:15.101055 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3a2ef-2e67-4556-8009-4bc4f1bba932 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:15.101875 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d3a2ef-2e67-4556-8009-4bc4f1bba932 None admin] RBAC: Adding query filter params (interface=admin, service_id=5e2fa717d1d64eaf98a52757a50ea6b9, region_id=RegionOne) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:15.102191 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d3a2ef-2e67-4556-8009-4bc4f1bba932 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:15.103609 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d3a2ef-2e67-4556-8009-4bc4f1bba932 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:15.109533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 496/991] 10.222.0.97 () {56 vars in 1249 bytes} [Tue Jul 26 14:31:14 2022] GET /identity/v3/endpoints?service_id=5e2fa717d1d64eaf98a52757a50ea6b9&interface=admin®ion_id=RegionOne => generated 193 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:15.115937 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d3a2ef-2e67-4556-8009-4bc4f1bba932 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:15.168856 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:15.173796 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05024f1-7b28-40a6-932f-34c212ba0eca None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:15.216092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cfe6be-2da8-43fb-b0af-6fa4b12a0c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:15.216719 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfe6be-2da8-43fb-b0af-6fa4b12a0c26 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:15.217000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfe6be-2da8-43fb-b0af-6fa4b12a0c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:15.217275 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfe6be-2da8-43fb-b0af-6fa4b12a0c26 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:15.217962 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cfe6be-2da8-43fb-b0af-6fa4b12a0c26 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:15.218268 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cfe6be-2da8-43fb-b0af-6fa4b12a0c26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:15.219666 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cfe6be-2da8-43fb-b0af-6fa4b12a0c26 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:15.224748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 496/992] 10.222.0.97 () {56 vars in 1112 bytes} [Tue Jul 26 14:31:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:15.230531 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cfe6be-2da8-43fb-b0af-6fa4b12a0c26 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:15.320468 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:15.341952 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d3a2ef-2e67-4556-8009-4bc4f1bba932 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:15.377333 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f80a6-11e8-4ff6-a379-547966dd86c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:15.377870 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f80a6-11e8-4ff6-a379-547966dd86c2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:15.378149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f80a6-11e8-4ff6-a379-547966dd86c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:15.378417 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f80a6-11e8-4ff6-a379-547966dd86c2 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:15.379223 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f80a6-11e8-4ff6-a379-547966dd86c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:15.382309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f80a6-11e8-4ff6-a379-547966dd86c2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:15.400591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 497/993] 10.222.0.97 () {56 vars in 1087 bytes} [Tue Jul 26 14:31:15 2022] GET /identity/v3/services => generated 1727 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:31:15.401818 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c75ca-aff3-48c7-98a3-8f9ca54019d6 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:15.402103 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c75ca-aff3-48c7-98a3-8f9ca54019d6 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:15.402374 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c75ca-aff3-48c7-98a3-8f9ca54019d6 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:15.403639 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 498/994] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:15.409485 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96caa2cf-185b-419c-9531-685cbbfb1da9 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:15.409821 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96caa2cf-185b-419c-9531-685cbbfb1da9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:15.410051 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96caa2cf-185b-419c-9531-685cbbfb1da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:15.422391 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cfe6be-2da8-43fb-b0af-6fa4b12a0c26 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:15.432194 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-96caa2cf-185b-419c-9531-685cbbfb1da9 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:15.433437 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-96caa2cf-185b-419c-9531-685cbbfb1da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:15.463601 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ee883-1e39-46e8-9084-8fd5a836d39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:15.464269 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ee883-1e39-46e8-9084-8fd5a836d39c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:15.464551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ee883-1e39-46e8-9084-8fd5a836d39c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:15.464824 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ee883-1e39-46e8-9084-8fd5a836d39c None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:15.465511 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ee883-1e39-46e8-9084-8fd5a836d39c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:15.465809 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ee883-1e39-46e8-9084-8fd5a836d39c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:15.468381 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ee883-1e39-46e8-9084-8fd5a836d39c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:15.476855 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 497/995] 10.222.0.97 () {56 vars in 1112 bytes} [Tue Jul 26 14:31:15 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:15.482610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ee883-1e39-46e8-9084-8fd5a836d39c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:15.535056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 499/996] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:15 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:15.560403 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96caa2cf-185b-419c-9531-685cbbfb1da9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:15.602157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:15.605250 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:15.629793 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ee883-1e39-46e8-9084-8fd5a836d39c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:15.666894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78960e7-ab15-4582-9352-ed5a5b987a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:15.667559 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78960e7-ab15-4582-9352-ed5a5b987a4f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:15.667847 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78960e7-ab15-4582-9352-ed5a5b987a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:15.668125 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78960e7-ab15-4582-9352-ed5a5b987a4f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:15.668911 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78960e7-ab15-4582-9352-ed5a5b987a4f None admin] RBAC: Adding query filter params (interface=public, service_id=724e8a4f7e8641fdabc8270663c789ca, region_id=RegionOne) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:15.669225 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78960e7-ab15-4582-9352-ed5a5b987a4f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:15.670605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78960e7-ab15-4582-9352-ed5a5b987a4f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:15.683439 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 498/997] 10.222.0.97 () {56 vars in 1251 bytes} [Tue Jul 26 14:31:15 2022] GET /identity/v3/endpoints?service_id=724e8a4f7e8641fdabc8270663c789ca&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:15.684654 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78960e7-ab15-4582-9352-ed5a5b987a4f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:15.692569 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96caa2cf-185b-419c-9531-685cbbfb1da9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:15.731681 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca511860-6f42-44e4-b8e9-fe8b9ca7fb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:15.732378 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca511860-6f42-44e4-b8e9-fe8b9ca7fb06 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:15.732714 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca511860-6f42-44e4-b8e9-fe8b9ca7fb06 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:15.733045 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca511860-6f42-44e4-b8e9-fe8b9ca7fb06 None admin] PATH_INFO: `/v3/services/image` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:15.734069 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca511860-6f42-44e4-b8e9-fe8b9ca7fb06 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:15.735212 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca511860-6f42-44e4-b8e9-fe8b9ca7fb06 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:15.745742 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-ca511860-6f42-44e4-b8e9-fe8b9ca7fb06 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 14:31:15.745742 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:15.745742 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:31:15.745742 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:31:15.745742 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:31:15.745742 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:31:15.745742 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:31:15.745742 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:31:15.745742 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 14:31:15.745742 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:15.745742 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:31:15.745742 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:15.745742 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:15.745742 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:15.745742 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:15.745742 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:15.745742 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:15.752071 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:15.752071 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:15.752071 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:15.752071 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:15.752071 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:15.752071 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:15.752071 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:31:15.752071 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:31:15.752071 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:31:15.752071 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:31:15.752071 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:15.752071 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:15.752071 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:15.752071 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:15.752071 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:15.752071 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:15.753127 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:15.753127 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:15.753127 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:15.753127 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:15.753127 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:15.753127 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:15.753127 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:15.753127 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:15.753127 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:15.753127 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:15.753127 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:31:15.753127 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:31:15.753127 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 14:31:15.753127 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:15.753127 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 500/998] 10.222.0.97 () {56 vars in 1105 bytes} [Tue Jul 26 14:31:15 2022] GET /identity/v3/services/image => generated 86 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:31:15.754121 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca511860-6f42-44e4-b8e9-fe8b9ca7fb06 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:15.804441 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78960e7-ab15-4582-9352-ed5a5b987a4f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:15.843859 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563e2a6-d7ad-4e69-824d-6fdc39aed7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:15.844646 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563e2a6-d7ad-4e69-824d-6fdc39aed7f0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:15.845315 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563e2a6-d7ad-4e69-824d-6fdc39aed7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:15.845594 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563e2a6-d7ad-4e69-824d-6fdc39aed7f0 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:15.846414 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2563e2a6-d7ad-4e69-824d-6fdc39aed7f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:15.847809 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2563e2a6-d7ad-4e69-824d-6fdc39aed7f0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:15.863442 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 499/999] 10.222.0.97 () {56 vars in 1087 bytes} [Tue Jul 26 14:31:15 2022] GET /identity/v3/services => generated 1727 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:31:15.880394 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:15.883890 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca511860-6f42-44e4-b8e9-fe8b9ca7fb06 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:15.923042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7c180c-8ee3-4f62-81c7-7e49a28d4074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:15.923718 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c180c-8ee3-4f62-81c7-7e49a28d4074 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:15.924010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c180c-8ee3-4f62-81c7-7e49a28d4074 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:15.924289 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c180c-8ee3-4f62-81c7-7e49a28d4074 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:15.924989 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7c180c-8ee3-4f62-81c7-7e49a28d4074 None admin] RBAC: Adding query filter params (name=image) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:15.925299 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7c180c-8ee3-4f62-81c7-7e49a28d4074 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:15.926776 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7c180c-8ee3-4f62-81c7-7e49a28d4074 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:15.937028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 501/1000] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:31:15 2022] GET /identity/v3/services?name=image => generated 122 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:15.938305 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563e2a6-d7ad-4e69-824d-6fdc39aed7f0 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:15.941342 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:16.062744 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563e2a6-d7ad-4e69-824d-6fdc39aed7f0 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:16.111328 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379bbef1-0701-410c-8715-a96b30afc296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:16.112341 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379bbef1-0701-410c-8715-a96b30afc296 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:16.112812 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379bbef1-0701-410c-8715-a96b30afc296 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:16.113288 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379bbef1-0701-410c-8715-a96b30afc296 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:16.114173 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379bbef1-0701-410c-8715-a96b30afc296 None admin] RBAC: Adding query filter params (type=image) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:16.114672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379bbef1-0701-410c-8715-a96b30afc296 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:16.116256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379bbef1-0701-410c-8715-a96b30afc296 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:16.121877 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 500/1001] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:31:15 2022] GET /identity/v3/services?type=image => generated 122 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:17.481121 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1d762-7647-410a-9652-bf1a103cf7fe None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:17.481660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1d762-7647-410a-9652-bf1a103cf7fe None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:17.481660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1d762-7647-410a-9652-bf1a103cf7fe None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:17.483100 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 502/1002] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:17.489013 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c8c0c-ebd0-4b5c-9dc7-46b2870e1bf7 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:17.489394 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c8c0c-ebd0-4b5c-9dc7-46b2870e1bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:17.489574 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c8c0c-ebd0-4b5c-9dc7-46b2870e1bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:17.507755 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-bd1c8c0c-ebd0-4b5c-9dc7-46b2870e1bf7 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:17.508929 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-bd1c8c0c-ebd0-4b5c-9dc7-46b2870e1bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:17.571058 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:17.579713 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 501/1003] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:17.602968 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1d762-7647-410a-9652-bf1a103cf7fe None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:17.705565 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1d762-7647-410a-9652-bf1a103cf7fe None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:17.745817 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23f9e9-0d7b-4f9c-a224-c9967a82201b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:17.746550 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23f9e9-0d7b-4f9c-a224-c9967a82201b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:17.747003 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23f9e9-0d7b-4f9c-a224-c9967a82201b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:17.747901 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23f9e9-0d7b-4f9c-a224-c9967a82201b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:17.748385 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23f9e9-0d7b-4f9c-a224-c9967a82201b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:17.750154 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23f9e9-0d7b-4f9c-a224-c9967a82201b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:17.762953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-ce23f9e9-0d7b-4f9c-a224-c9967a82201b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 14:31:17.762953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:17.762953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:31:17.762953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:31:17.762953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:31:17.762953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:31:17.762953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:31:17.762953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:31:17.762953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 14:31:17.762953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:17.762953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:31:17.762953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:17.762953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:17.762953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:17.762953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:17.762953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:17.762953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:17.764707 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:17.764707 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:17.764707 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:17.764707 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:17.764707 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:17.764707 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:17.764707 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:31:17.764707 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:31:17.764707 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:31:17.764707 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:31:17.764707 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:17.764707 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:17.764707 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:17.764707 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:17.764707 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:17.764707 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:17.765822 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:17.765822 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:17.765822 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:17.765822 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:17.765822 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:17.765822 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:17.765822 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:17.765822 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:17.765822 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:17.765822 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:17.765822 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:31:17.765822 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:31:17.765822 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 14:31:17.765822 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:17.767739 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 503/1004] 10.222.0.97 () {56 vars in 1126 bytes} [Tue Jul 26 14:31:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:31:17.771838 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c8c0c-ebd0-4b5c-9dc7-46b2870e1bf7 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:17.874476 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c8c0c-ebd0-4b5c-9dc7-46b2870e1bf7 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:17.909226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33946f4-7388-43eb-b4f7-0ccf00338256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:17.909857 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33946f4-7388-43eb-b4f7-0ccf00338256 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:17.909972 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b3d0e-a02f-421b-9dd1-bdf31e13cb36 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:17.910148 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33946f4-7388-43eb-b4f7-0ccf00338256 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:17.910959 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33946f4-7388-43eb-b4f7-0ccf00338256 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:17.911379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b3d0e-a02f-421b-9dd1-bdf31e13cb36 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:17.911674 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33946f4-7388-43eb-b4f7-0ccf00338256 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:17.911779 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b3d0e-a02f-421b-9dd1-bdf31e13cb36 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:17.911976 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33946f4-7388-43eb-b4f7-0ccf00338256 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:17.913181 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 504/1005] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:31:17.913399 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33946f4-7388-43eb-b4f7-0ccf00338256 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:17.918115 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09fc3a5-a4cc-4d69-9fde-69d4c9b54ea5 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:17.918403 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09fc3a5-a4cc-4d69-9fde-69d4c9b54ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:17.918684 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09fc3a5-a4cc-4d69-9fde-69d4c9b54ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:17.923511 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 502/1006] 10.222.0.97 () {56 vars in 1122 bytes} [Tue Jul 26 14:31:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:17.924628 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33946f4-7388-43eb-b4f7-0ccf00338256 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:17.940197 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a09fc3a5-a4cc-4d69-9fde-69d4c9b54ea5 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:17.941490 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a09fc3a5-a4cc-4d69-9fde-69d4c9b54ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:17.965719 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:17.994517 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:18.016194 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 505/1007] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 14:31:18.039297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09fc3a5-a4cc-4d69-9fde-69d4c9b54ea5 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:18.042066 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33946f4-7388-43eb-b4f7-0ccf00338256 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:18.081659 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa8be3-080f-49bc-a6d5-e9a46de98a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:18.082308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa8be3-080f-49bc-a6d5-e9a46de98a43 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:18.082592 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa8be3-080f-49bc-a6d5-e9a46de98a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:18.082863 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa8be3-080f-49bc-a6d5-e9a46de98a43 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:18.083683 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa8be3-080f-49bc-a6d5-e9a46de98a43 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:18.083992 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa8be3-080f-49bc-a6d5-e9a46de98a43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:18.085349 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa8be3-080f-49bc-a6d5-e9a46de98a43 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:18.094980 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 503/1008] 10.222.0.97 () {56 vars in 1122 bytes} [Tue Jul 26 14:31:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:18.096016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa8be3-080f-49bc-a6d5-e9a46de98a43 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:18.149930 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09fc3a5-a4cc-4d69-9fde-69d4c9b54ea5 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:18.184798 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e235d-63bf-4c8a-bb2c-aa921abc830e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:18.189464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e235d-63bf-4c8a-bb2c-aa921abc830e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:18.189464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e235d-63bf-4c8a-bb2c-aa921abc830e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:18.189464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e235d-63bf-4c8a-bb2c-aa921abc830e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:18.189464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e235d-63bf-4c8a-bb2c-aa921abc830e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:18.189464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e235d-63bf-4c8a-bb2c-aa921abc830e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:18.197131 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-740e235d-63bf-4c8a-bb2c-aa921abc830e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 14:31:18.197131 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:18.197131 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:31:18.197131 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:31:18.197131 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:31:18.197131 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:31:18.197131 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:31:18.197131 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:31:18.197131 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 14:31:18.197131 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:18.197131 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:31:18.197131 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:18.197131 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:18.197131 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:18.197131 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:18.197131 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:18.197131 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:18.198340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:18.198340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:18.198340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:18.198340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:18.198340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:18.198340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:18.198340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:31:18.198340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:31:18.198340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:31:18.198340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:31:18.198340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:18.198340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:18.198340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:18.198340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:18.198340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:18.198340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:18.199433 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:18.199433 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:18.199433 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:18.199433 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:18.199433 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:18.199433 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:18.199433 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:18.199433 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:18.199433 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:18.199433 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:18.199433 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:31:18.199433 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:31:18.199433 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 14:31:18.199433 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:18.199433 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 506/1009] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:31:18 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:31:18.200586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82763d26-81ff-4ffc-893c-d5b464059fc1 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:18.200875 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82763d26-81ff-4ffc-893c-d5b464059fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:18.201153 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82763d26-81ff-4ffc-893c-d5b464059fc1 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:18.202421 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 507/1010] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:18.204796 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82763d26-81ff-4ffc-893c-d5b464059fc1 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:18.208593 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa8be3-080f-49bc-a6d5-e9a46de98a43 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:18.230333 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:18.232632 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:18.243072 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb615e-9765-4f67-be79-d1a7555f0096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:18.243726 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb615e-9765-4f67-be79-d1a7555f0096 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:18.244011 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb615e-9765-4f67-be79-d1a7555f0096 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:18.244289 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb615e-9765-4f67-be79-d1a7555f0096 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:18.245031 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb615e-9765-4f67-be79-d1a7555f0096 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:18.246406 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb615e-9765-4f67-be79-d1a7555f0096 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:18.306267 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82763d26-81ff-4ffc-893c-d5b464059fc1 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:18.338996 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a1cdd-9abf-4679-854e-857a6fe80797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:18.339632 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a1cdd-9abf-4679-854e-857a6fe80797 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:18.339919 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a1cdd-9abf-4679-854e-857a6fe80797 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:18.340194 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a1cdd-9abf-4679-854e-857a6fe80797 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:18.340866 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a1cdd-9abf-4679-854e-857a6fe80797 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:18.341172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a1cdd-9abf-4679-854e-857a6fe80797 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:18.342603 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a1cdd-9abf-4679-854e-857a6fe80797 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:18.347472 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 508/1011] 10.222.0.97 () {56 vars in 1112 bytes} [Tue Jul 26 14:31:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:18.349004 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ce24b-3f65-4734-b0af-59e9b748e3be None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:18.349288 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ce24b-3f65-4734-b0af-59e9b748e3be None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:18.349563 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ce24b-3f65-4734-b0af-59e9b748e3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:18.373167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 504/1012] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:31:18 2022] POST /identity/v3/endpoints => generated 325 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:31:18.373167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb615e-9765-4f67-be79-d1a7555f0096 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:18.373502 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-df3ce24b-3f65-4734-b0af-59e9b748e3be None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:18.373502 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-df3ce24b-3f65-4734-b0af-59e9b748e3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:18.459454 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 509/1013] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:18 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:18.478833 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ce24b-3f65-4734-b0af-59e9b748e3be None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:18.488196 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb615e-9765-4f67-be79-d1a7555f0096 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:18.494304 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:18.529726 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b6be8-eece-4c3c-8eec-8f6103445656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:18.530509 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b6be8-eece-4c3c-8eec-8f6103445656 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:18.530801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b6be8-eece-4c3c-8eec-8f6103445656 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:18.531085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b6be8-eece-4c3c-8eec-8f6103445656 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:18.532200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b6be8-eece-4c3c-8eec-8f6103445656 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:18.532200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b6be8-eece-4c3c-8eec-8f6103445656 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:18.533520 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b6be8-eece-4c3c-8eec-8f6103445656 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:18.540607 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 505/1014] 10.222.0.97 () {56 vars in 1112 bytes} [Tue Jul 26 14:31:18 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:18.545658 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b6be8-eece-4c3c-8eec-8f6103445656 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:18.597394 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:18.601782 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ce24b-3f65-4734-b0af-59e9b748e3be None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:18.641887 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3c87d-08e3-45c8-b77a-a571f9ad64fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:18.642497 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3c87d-08e3-45c8-b77a-a571f9ad64fb None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:18.642782 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3c87d-08e3-45c8-b77a-a571f9ad64fb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:18.643061 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3c87d-08e3-45c8-b77a-a571f9ad64fb None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:18.643837 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3c87d-08e3-45c8-b77a-a571f9ad64fb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:18.645292 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3c87d-08e3-45c8-b77a-a571f9ad64fb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:18.662714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b6be8-eece-4c3c-8eec-8f6103445656 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:18.700781 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a63d7f-2866-473a-9bfa-5c9408f56586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:18.701759 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a63d7f-2866-473a-9bfa-5c9408f56586 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:18.702047 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a63d7f-2866-473a-9bfa-5c9408f56586 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:18.702410 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a63d7f-2866-473a-9bfa-5c9408f56586 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:18.703208 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a63d7f-2866-473a-9bfa-5c9408f56586 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:18.704564 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a63d7f-2866-473a-9bfa-5c9408f56586 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:18.742403 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 510/1015] 10.222.0.97 () {60 vars in 1141 bytes} [Tue Jul 26 14:31:18 2022] POST /identity/v3/services => generated 242 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:31:18.816497 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 506/1016] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:31:18 2022] POST /identity/v3/endpoints => generated 334 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:31:20.427368 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbe441-17eb-4345-baec-0f4ad0fdf3b7 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:20.427903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbe441-17eb-4345-baec-0f4ad0fdf3b7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:20.427903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbe441-17eb-4345-baec-0f4ad0fdf3b7 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:20.429241 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 511/1017] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:20.434496 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e3657-5fd4-4765-a6cb-4ac0e62759ba None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:20.434912 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e3657-5fd4-4765-a6cb-4ac0e62759ba None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:20.435135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e3657-5fd4-4765-a6cb-4ac0e62759ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:20.453266 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-945e3657-5fd4-4765-a6cb-4ac0e62759ba None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:20.454463 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-945e3657-5fd4-4765-a6cb-4ac0e62759ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:20.536149 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 507/1018] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:20 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:20.559144 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cbe441-17eb-4345-baec-0f4ad0fdf3b7 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:20.635060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:20.670401 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cbe441-17eb-4345-baec-0f4ad0fdf3b7 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:20.707915 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94040066-9e47-4cc1-af88-29e8ba16f3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:20.707915 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94040066-9e47-4cc1-af88-29e8ba16f3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:20.707915 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94040066-9e47-4cc1-af88-29e8ba16f3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:20.707915 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94040066-9e47-4cc1-af88-29e8ba16f3f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:20.708527 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94040066-9e47-4cc1-af88-29e8ba16f3f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:20.709851 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94040066-9e47-4cc1-af88-29e8ba16f3f5 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:20.711695 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 512/1019] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:31:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:20.717374 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e3657-5fd4-4765-a6cb-4ac0e62759ba None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:20.749597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:20.796806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9fbc1c-f38f-4d60-b732-a850d7372580 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:20.797154 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9fbc1c-f38f-4d60-b732-a850d7372580 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:20.797366 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9fbc1c-f38f-4d60-b732-a850d7372580 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:20.798996 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 513/1020] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:31:20.804811 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a072b5-1f3b-43bb-99c4-259ea442835c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:20.805109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a072b5-1f3b-43bb-99c4-259ea442835c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:20.805377 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a072b5-1f3b-43bb-99c4-259ea442835c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:20.824428 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-46a072b5-1f3b-43bb-99c4-259ea442835c None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:20.825638 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-46a072b5-1f3b-43bb-99c4-259ea442835c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:20.844692 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e3657-5fd4-4765-a6cb-4ac0e62759ba None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:20.883868 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c2df8a-8b20-4ba8-b3c6-340075b57cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:20.884509 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2df8a-8b20-4ba8-b3c6-340075b57cbc None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:20.884798 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2df8a-8b20-4ba8-b3c6-340075b57cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:20.885071 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2df8a-8b20-4ba8-b3c6-340075b57cbc None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:20.885778 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c2df8a-8b20-4ba8-b3c6-340075b57cbc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:20.887259 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c2df8a-8b20-4ba8-b3c6-340075b57cbc None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:20.902878 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 514/1021] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:20 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 14:31:20.925482 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a072b5-1f3b-43bb-99c4-259ea442835c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:20.986244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 508/1022] 10.222.0.97 () {60 vars in 1141 bytes} [Tue Jul 26 14:31:20 2022] POST /identity/v3/projects => generated 311 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:31:20.986244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6156aba-8cde-4859-bf12-449dc0515c14 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:20.986244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6156aba-8cde-4859-bf12-449dc0515c14 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:20.986244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6156aba-8cde-4859-bf12-449dc0515c14 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:20.986244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 509/1023] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:20.996234 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c7f27-4cb3-4da9-add5-3b833669940f None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:20.996613 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c7f27-4cb3-4da9-add5-3b833669940f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:20.996808 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c7f27-4cb3-4da9-add5-3b833669940f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:21.002435 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:21.017571 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c86c7f27-4cb3-4da9-add5-3b833669940f None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:21.018835 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c86c7f27-4cb3-4da9-add5-3b833669940f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:21.039469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a072b5-1f3b-43bb-99c4-259ea442835c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:21.095303 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2895a534-aaf7-4f9c-99c2-735e3edf9e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:21.096721 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895a534-aaf7-4f9c-99c2-735e3edf9e0b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:21.097239 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895a534-aaf7-4f9c-99c2-735e3edf9e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:21.098191 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895a534-aaf7-4f9c-99c2-735e3edf9e0b None admin] PATH_INFO: `/v3/services/image` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:21.099165 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2895a534-aaf7-4f9c-99c2-735e3edf9e0b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:21.101914 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2895a534-aaf7-4f9c-99c2-735e3edf9e0b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:21.115695 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-2895a534-aaf7-4f9c-99c2-735e3edf9e0b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 14:31:21.115695 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:21.115695 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:31:21.115695 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:31:21.115695 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:31:21.115695 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:31:21.115695 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:31:21.115695 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:31:21.115695 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 14:31:21.115695 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:21.115695 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:31:21.115695 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:21.115695 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:21.115695 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:21.115695 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:21.115695 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:21.115695 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:21.116920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:21.116920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:21.116920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:21.116920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:21.116920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:21.116920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:21.116920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:31:21.116920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:31:21.116920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:31:21.116920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:31:21.116920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:21.116920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:21.116920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:21.116920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:21.116920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:21.116920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:21.118025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:21.118025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:21.118025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:21.118025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:21.118025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:21.118025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:21.118025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:21.118025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:21.118025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:21.118025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:21.118025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:31:21.118025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:31:21.118025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 14:31:21.118025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:21.125597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 510/1024] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:20 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:21.125778 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 515/1025] 10.222.0.97 () {56 vars in 1105 bytes} [Tue Jul 26 14:31:20 2022] GET /identity/v3/services/image => generated 86 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:31:21.127419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c7f27-4cb3-4da9-add5-3b833669940f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:21.149807 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:21.156123 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2895a534-aaf7-4f9c-99c2-735e3edf9e0b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:21.278205 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c7f27-4cb3-4da9-add5-3b833669940f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:21.301528 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2895a534-aaf7-4f9c-99c2-735e3edf9e0b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:21.315046 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038f855-541d-4dff-9680-9fb26539641f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:21.320089 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038f855-541d-4dff-9680-9fb26539641f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:21.320089 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038f855-541d-4dff-9680-9fb26539641f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:21.320089 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038f855-541d-4dff-9680-9fb26539641f None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:21.320089 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8038f855-541d-4dff-9680-9fb26539641f None admin] RBAC: Adding query filter params (name=image) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:21.320089 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8038f855-541d-4dff-9680-9fb26539641f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:21.320089 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8038f855-541d-4dff-9680-9fb26539641f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:21.328512 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 511/1026] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:31:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:21.330008 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038f855-541d-4dff-9680-9fb26539641f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:21.345331 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5edc05-0ce9-4cf2-ae08-03425a8bf094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:21.345971 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5edc05-0ce9-4cf2-ae08-03425a8bf094 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:21.346256 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5edc05-0ce9-4cf2-ae08-03425a8bf094 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:21.346547 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5edc05-0ce9-4cf2-ae08-03425a8bf094 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:21.349114 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:21.359297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5edc05-0ce9-4cf2-ae08-03425a8bf094 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:31:21.364218 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-df5edc05-0ce9-4cf2-ae08-03425a8bf094 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 14:31:21.364218 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:21.364218 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:21.364218 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:21.364218 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:21.364218 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:21.364218 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:21.364218 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:21.364218 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:21.364218 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:21.364218 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:21.364218 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:21.364218 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:31:21.364218 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:31:21.364218 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:31:21.364218 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:31:21.365170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:31:21.365170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:31:21.365170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:31:21.365170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:31:21.365170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:21.365170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:21.365170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:21.365170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:21.365170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:21.365170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:21.365170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:21.365170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:21.365170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:21.365170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:21.365170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:21.365170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:21.366604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:21.366604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:21.366604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:21.366604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:21.366604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:31:21.366604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:31:21.366604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:31:21.366604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:31:21.366604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:31:21.366604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:31:21.366604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 14:31:21.366604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:21.366604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 516/1027] 10.222.0.97 () {56 vars in 1120 bytes} [Tue Jul 26 14:31:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:31:21.371239 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5edc05-0ce9-4cf2-ae08-03425a8bf094 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:21.420718 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:21.446178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038f855-541d-4dff-9680-9fb26539641f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:21.477780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5edc05-0ce9-4cf2-ae08-03425a8bf094 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:21.482381 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8bde7-0e8c-487a-b39e-76bc88858169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:21.483011 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8bde7-0e8c-487a-b39e-76bc88858169 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:21.483305 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8bde7-0e8c-487a-b39e-76bc88858169 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:21.483609 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8bde7-0e8c-487a-b39e-76bc88858169 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:21.484281 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad8bde7-0e8c-487a-b39e-76bc88858169 None admin] RBAC: Adding query filter params (type=image) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:21.484583 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad8bde7-0e8c-487a-b39e-76bc88858169 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:21.485934 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad8bde7-0e8c-487a-b39e-76bc88858169 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:21.496582 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 512/1028] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:31:21 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:21.497607 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8bde7-0e8c-487a-b39e-76bc88858169 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:21.521485 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce59042-0c88-4eb3-9e8f-b4bf2a500a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:21.522435 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce59042-0c88-4eb3-9e8f-b4bf2a500a20 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:21.522906 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce59042-0c88-4eb3-9e8f-b4bf2a500a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:21.523374 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce59042-0c88-4eb3-9e8f-b4bf2a500a20 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:21.524287 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce59042-0c88-4eb3-9e8f-b4bf2a500a20 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:21.524595 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce59042-0c88-4eb3-9e8f-b4bf2a500a20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:21.525949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce59042-0c88-4eb3-9e8f-b4bf2a500a20 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:21.539516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 517/1029] 10.222.0.97 () {56 vars in 1115 bytes} [Tue Jul 26 14:31:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:21.540147 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce59042-0c88-4eb3-9e8f-b4bf2a500a20 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:21.601858 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:21.610914 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8bde7-0e8c-487a-b39e-76bc88858169 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:21.646265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb767ff-cfe9-4577-839a-cf20509bb713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:21.646908 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb767ff-cfe9-4577-839a-cf20509bb713 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:21.647189 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb767ff-cfe9-4577-839a-cf20509bb713 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:21.647491 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb767ff-cfe9-4577-839a-cf20509bb713 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:21.647594 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce59042-0c88-4eb3-9e8f-b4bf2a500a20 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:21.648236 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb767ff-cfe9-4577-839a-cf20509bb713 None admin] RBAC: Adding query filter params (interface=public, service_id=b35cd2b6381a43d9b018a8c292763443, region_id=RegionOne) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:21.648543 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb767ff-cfe9-4577-839a-cf20509bb713 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:21.649899 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb767ff-cfe9-4577-839a-cf20509bb713 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:21.655041 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 513/1030] 10.222.0.97 () {56 vars in 1251 bytes} [Tue Jul 26 14:31:21 2022] GET /identity/v3/endpoints?service_id=b35cd2b6381a43d9b018a8c292763443&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:21.660423 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb767ff-cfe9-4577-839a-cf20509bb713 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:21.663909 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:21.682717 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0b38c-396b-4930-b2d2-b5875cfedafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:21.683617 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0b38c-396b-4930-b2d2-b5875cfedafc None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:21.684044 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0b38c-396b-4930-b2d2-b5875cfedafc None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:21.684482 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0b38c-396b-4930-b2d2-b5875cfedafc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:21.696146 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc0b38c-396b-4930-b2d2-b5875cfedafc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:21.697571 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc0b38c-396b-4930-b2d2-b5875cfedafc None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:21.712839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-3cc0b38c-396b-4930-b2d2-b5875cfedafc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:31:21.712839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:21.712839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:21.712839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:21.712839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:21.712839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:21.712839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:21.712839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:21.712839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:21.712839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:21.712839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:21.712839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:21.712839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:31:21.712839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:31:21.712839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:31:21.713973 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:31:21.713973 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:21.713973 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:21.713973 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:21.713973 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:21.713973 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:21.713973 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:21.713973 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:21.713973 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:21.713973 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:21.713973 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:21.713973 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:21.713973 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:21.713973 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:21.713973 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:21.713973 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:21.713973 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:21.714978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:31:21.714978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:31:21.714978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:31:21.714978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:21.714978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 518/1031] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:31:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:31:21.719549 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0b38c-396b-4930-b2d2-b5875cfedafc None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:21.777596 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb767ff-cfe9-4577-839a-cf20509bb713 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:21.816390 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f325f52-3930-478e-8f1a-4e43a98ca3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:21.816933 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f325f52-3930-478e-8f1a-4e43a98ca3aa None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:21.817216 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f325f52-3930-478e-8f1a-4e43a98ca3aa None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:21.817500 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f325f52-3930-478e-8f1a-4e43a98ca3aa None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:21.818367 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f325f52-3930-478e-8f1a-4e43a98ca3aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:21.819554 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f325f52-3930-478e-8f1a-4e43a98ca3aa None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:21.826579 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 514/1032] 10.222.0.97 () {56 vars in 1087 bytes} [Tue Jul 26 14:31:21 2022] GET /identity/v3/services => generated 1957 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:31:21.826966 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0b38c-396b-4930-b2d2-b5875cfedafc None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:21.837436 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:21.864957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7559c2-862c-4135-afe1-c4941acef6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:21.865585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7559c2-862c-4135-afe1-c4941acef6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:21.865868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7559c2-862c-4135-afe1-c4941acef6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:21.866141 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7559c2-862c-4135-afe1-c4941acef6c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:21.866835 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7559c2-862c-4135-afe1-c4941acef6c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:21.867139 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7559c2-862c-4135-afe1-c4941acef6c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:21.868548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7559c2-862c-4135-afe1-c4941acef6c9 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:21.887570 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f325f52-3930-478e-8f1a-4e43a98ca3aa None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:21.887926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 519/1033] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:31:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:21.965406 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:22.002364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f325f52-3930-478e-8f1a-4e43a98ca3aa None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:22.038165 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8dbf9-1b66-42c2-9107-b635a80465a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:22.039091 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8dbf9-1b66-42c2-9107-b635a80465a9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:22.039668 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8dbf9-1b66-42c2-9107-b635a80465a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:22.040075 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8dbf9-1b66-42c2-9107-b635a80465a9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:22.048080 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8dbf9-1b66-42c2-9107-b635a80465a9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:22.049712 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8dbf9-1b66-42c2-9107-b635a80465a9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:22.061979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-34d8dbf9-1b66-42c2-9107-b635a80465a9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:31:22.061979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:22.061979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:22.061979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:22.061979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:22.061979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:22.061979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:22.061979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:22.061979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:22.061979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:22.061979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:22.061979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:22.061979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:31:22.061979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:31:22.061979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:31:22.061979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:31:22.062917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:22.062917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:22.062917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:31:22.062917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:31:22.062917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:31:22.062917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:31:22.062917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:22.062917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:22.062917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:22.062917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:22.062917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:22.062917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:22.062917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:22.062917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:22.062917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:22.062917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:22.062917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:22.062917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:22.064231 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:22.064231 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:22.064231 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:31:22.064231 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:31:22.064231 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:31:22.064231 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:31:22.064231 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:31:22.064231 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:31:22.064231 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:31:22.064231 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:22.064231 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 515/1034] 10.222.0.97 () {56 vars in 1093 bytes} [Tue Jul 26 14:31:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:31:22.069347 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7559c2-862c-4135-afe1-c4941acef6c9 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:22.175647 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7559c2-862c-4135-afe1-c4941acef6c9 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:22.211843 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33accd-95d3-464b-9d2c-aa46d2979c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:22.212456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33accd-95d3-464b-9d2c-aa46d2979c25 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:22.212736 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33accd-95d3-464b-9d2c-aa46d2979c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:22.213005 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33accd-95d3-464b-9d2c-aa46d2979c25 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:22.213785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c33accd-95d3-464b-9d2c-aa46d2979c25 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:22.214086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c33accd-95d3-464b-9d2c-aa46d2979c25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:22.215530 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c33accd-95d3-464b-9d2c-aa46d2979c25 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:22.233676 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 520/1035] 10.222.0.97 () {56 vars in 1133 bytes} [Tue Jul 26 14:31:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:22.239147 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8dbf9-1b66-42c2-9107-b635a80465a9 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:22.343830 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8dbf9-1b66-42c2-9107-b635a80465a9 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:22.370889 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:22.379353 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4f3d7-4479-42e0-a00c-90c062a303f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:22.380012 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4f3d7-4479-42e0-a00c-90c062a303f0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:22.388951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4f3d7-4479-42e0-a00c-90c062a303f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:22.388951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4f3d7-4479-42e0-a00c-90c062a303f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:22.447714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4f3d7-4479-42e0-a00c-90c062a303f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:22.449168 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4f3d7-4479-42e0-a00c-90c062a303f0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:22.548206 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 516/1036] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:31:22 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:31:22.554635 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33accd-95d3-464b-9d2c-aa46d2979c25 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:22.582524 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:22.657597 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33accd-95d3-464b-9d2c-aa46d2979c25 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:22.691494 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0fd282-500a-4019-b1c1-3f4528dd9069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:22.692136 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0fd282-500a-4019-b1c1-3f4528dd9069 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:22.692421 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0fd282-500a-4019-b1c1-3f4528dd9069 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:22.692697 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0fd282-500a-4019-b1c1-3f4528dd9069 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:22.703043 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0fd282-500a-4019-b1c1-3f4528dd9069 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:22.704507 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0fd282-500a-4019-b1c1-3f4528dd9069 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:22.719487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-db0fd282-500a-4019-b1c1-3f4528dd9069 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:31:22.719487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:22.719487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:22.719487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:22.719487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:22.719487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:22.719487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:22.719487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:22.719487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:22.719487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:22.719487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:22.719487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:22.719487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:31:22.719487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:31:22.719487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:31:22.720388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:31:22.720388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:22.720388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:22.720388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:22.720388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:22.720388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:22.720388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:22.720388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:22.720388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:22.720388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:22.720388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:22.720388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:22.720388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:22.720388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:22.720388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:22.720388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:22.720388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:22.721273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:31:22.721273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:31:22.721273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:31:22.721273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:22.721273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 521/1037] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:31:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:31:22.727918 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4f3d7-4479-42e0-a00c-90c062a303f0 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:22.755779 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:22.829765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4f3d7-4479-42e0-a00c-90c062a303f0 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:22.866999 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b631bc19-7325-4463-8ac0-cc08dbf7676d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:22.867674 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631bc19-7325-4463-8ac0-cc08dbf7676d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:22.867957 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631bc19-7325-4463-8ac0-cc08dbf7676d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:22.868233 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631bc19-7325-4463-8ac0-cc08dbf7676d None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:22.868924 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b631bc19-7325-4463-8ac0-cc08dbf7676d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:22.869229 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b631bc19-7325-4463-8ac0-cc08dbf7676d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:22.870597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b631bc19-7325-4463-8ac0-cc08dbf7676d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:22.886618 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 517/1038] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:31:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:22.887901 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0fd282-500a-4019-b1c1-3f4528dd9069 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:22.973228 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:22.994966 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0fd282-500a-4019-b1c1-3f4528dd9069 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:23.030559 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3f76e-7c42-4c46-9477-9f234a527869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:23.031203 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3f76e-7c42-4c46-9477-9f234a527869 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:23.031515 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3f76e-7c42-4c46-9477-9f234a527869 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:23.031795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3f76e-7c42-4c46-9477-9f234a527869 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:23.042999 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c3f76e-7c42-4c46-9477-9f234a527869 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:23.044466 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c3f76e-7c42-4c46-9477-9f234a527869 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:23.056396 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-48c3f76e-7c42-4c46-9477-9f234a527869 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:31:23.056396 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:23.056396 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:23.056396 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:23.056396 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:23.056396 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:23.056396 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:23.056396 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:23.056396 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:23.056396 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:23.056396 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:23.056396 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:23.056396 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:31:23.056396 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:31:23.056396 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:31:23.057277 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:31:23.057277 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:23.057277 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:23.057277 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:23.057277 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:23.057277 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:23.057277 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:23.057277 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:23.057277 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:23.057277 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:23.057277 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:23.057277 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:23.057277 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:23.057277 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:23.057277 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:23.057277 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:23.057277 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:23.058183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:31:23.058183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:31:23.058183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:31:23.058183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:31:23.058183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:31:23.058183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:31:23.058183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:31:23.058183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:23.059719 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 522/1039] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:31:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:31:23.063338 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b631bc19-7325-4463-8ac0-cc08dbf7676d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:23.087060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a68be-ca46-4896-845b-76aa4582459f None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:23.087346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a68be-ca46-4896-845b-76aa4582459f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:23.087667 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a68be-ca46-4896-845b-76aa4582459f None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:23.089141 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 523/1040] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:31:23.093904 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cacf558-fd16-476b-8c40-9bd2e758f128 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:23.094195 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cacf558-fd16-476b-8c40-9bd2e758f128 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:23.094479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cacf558-fd16-476b-8c40-9bd2e758f128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:23.114622 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-7cacf558-fd16-476b-8c40-9bd2e758f128 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:23.115851 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7cacf558-fd16-476b-8c40-9bd2e758f128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:23.158926 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:23.172274 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b631bc19-7325-4463-8ac0-cc08dbf7676d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:23.189394 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 524/1041] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 14:31:23.207968 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c3651-6ae7-43f8-90e8-6ba75f89661a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:23.208627 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c3651-6ae7-43f8-90e8-6ba75f89661a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:23.208913 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c3651-6ae7-43f8-90e8-6ba75f89661a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:23.209189 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c3651-6ae7-43f8-90e8-6ba75f89661a None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:23.209939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939c3651-6ae7-43f8-90e8-6ba75f89661a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:23.210247 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939c3651-6ae7-43f8-90e8-6ba75f89661a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:23.211707 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939c3651-6ae7-43f8-90e8-6ba75f89661a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:23.218668 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cacf558-fd16-476b-8c40-9bd2e758f128 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:23.230521 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 518/1042] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:31:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:23.231852 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c3651-6ae7-43f8-90e8-6ba75f89661a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:23.287609 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:23.321303 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cacf558-fd16-476b-8c40-9bd2e758f128 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:23.343509 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c3651-6ae7-43f8-90e8-6ba75f89661a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:23.359425 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba604a9-a420-493f-be95-14b5fdfa720f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:23.360358 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba604a9-a420-493f-be95-14b5fdfa720f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:23.360644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba604a9-a420-493f-be95-14b5fdfa720f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:23.360917 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba604a9-a420-493f-be95-14b5fdfa720f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:23.362074 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba604a9-a420-493f-be95-14b5fdfa720f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:23.364205 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba604a9-a420-493f-be95-14b5fdfa720f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:23.367444 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 525/1043] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:31:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:23.371781 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba604a9-a420-493f-be95-14b5fdfa720f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:23.380748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03271569-03c4-46ca-9f42-473a179258c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:23.381370 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03271569-03c4-46ca-9f42-473a179258c5 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:23.381652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03271569-03c4-46ca-9f42-473a179258c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:23.381929 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03271569-03c4-46ca-9f42-473a179258c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:23.382714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03271569-03c4-46ca-9f42-473a179258c5 None admin] RBAC: Adding query filter params (role.id=a61134ee45dd49b3a20a8a55044e0e3c, scope.project.id=c4c8da3011294783b2180ade1bfd9366, user.id=266abef6cda5495b897170f92043aa03) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:23.383032 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03271569-03c4-46ca-9f42-473a179258c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:23.384481 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03271569-03c4-46ca-9f42-473a179258c5 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:23.390190 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 519/1044] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:31:23 2022] GET /identity/v3/role_assignments?user.id=266abef6cda5495b897170f92043aa03&scope.project.id=c4c8da3011294783b2180ade1bfd9366&role.id=a61134ee45dd49b3a20a8a55044e0e3c => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:23.495511 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba604a9-a420-493f-be95-14b5fdfa720f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:23.533019 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f253e-a051-4052-876f-a1fc7b739058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:23.533651 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f253e-a051-4052-876f-a1fc7b739058 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:23.533934 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f253e-a051-4052-876f-a1fc7b739058 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:23.534215 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f253e-a051-4052-876f-a1fc7b739058 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:23.534941 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545f253e-a051-4052-876f-a1fc7b739058 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:23.536405 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545f253e-a051-4052-876f-a1fc7b739058 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:23.545693 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:23.625876 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 526/1045] 10.222.0.97 () {60 vars in 1132 bytes} [Tue Jul 26 14:31:23 2022] POST /identity/v3/users => generated 285 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:31:23.860279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc75c7f-fe0c-4bda-923d-5796c6e6cba4 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:23.860810 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc75c7f-fe0c-4bda-923d-5796c6e6cba4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:23.861280 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc75c7f-fe0c-4bda-923d-5796c6e6cba4 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:23.863854 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 520/1046] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:23.870215 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090879e9-efc7-43fd-8a9f-afa1db173d8a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:23.870543 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090879e9-efc7-43fd-8a9f-afa1db173d8a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:23.870823 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090879e9-efc7-43fd-8a9f-afa1db173d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:23.889976 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-090879e9-efc7-43fd-8a9f-afa1db173d8a None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:23.891159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-090879e9-efc7-43fd-8a9f-afa1db173d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:23.961162 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 527/1047] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:23.983527 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc75c7f-fe0c-4bda-923d-5796c6e6cba4 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:23.998224 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:24.094563 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc75c7f-fe0c-4bda-923d-5796c6e6cba4 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:24.129096 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09042fc-2e73-40d7-a401-e3c2965b4d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:24.129652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09042fc-2e73-40d7-a401-e3c2965b4d58 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:24.129932 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09042fc-2e73-40d7-a401-e3c2965b4d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:24.130202 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09042fc-2e73-40d7-a401-e3c2965b4d58 None admin] PATH_INFO: `/v3/services/image` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:24.130819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09042fc-2e73-40d7-a401-e3c2965b4d58 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:24.132202 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09042fc-2e73-40d7-a401-e3c2965b4d58 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:24.141265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-a09042fc-2e73-40d7-a401-e3c2965b4d58 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 14:31:24.141265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:24.141265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:31:24.141265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:31:24.141265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:31:24.141265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:31:24.141265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:31:24.141265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:31:24.141265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 14:31:24.141265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:24.141265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:31:24.141265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:24.141265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:24.141265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:24.141265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:24.141265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:24.141265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:24.142270 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:24.142270 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:24.142270 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:24.142270 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:24.142270 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:24.142270 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:24.142270 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:31:24.142270 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:31:24.142270 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:31:24.142270 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:31:24.142270 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:24.142270 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:24.142270 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:24.142270 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:24.142270 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:24.142270 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:24.143102 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:24.143102 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:24.143102 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:24.143102 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:24.143102 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:24.143102 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:24.143102 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:24.143102 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:24.143102 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:24.143102 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:24.143102 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:31:24.143102 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:31:24.143102 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 14:31:24.143102 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:24.147477 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 521/1048] 10.222.0.97 () {56 vars in 1105 bytes} [Tue Jul 26 14:31:23 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:31:24.149021 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090879e9-efc7-43fd-8a9f-afa1db173d8a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:24.249002 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090879e9-efc7-43fd-8a9f-afa1db173d8a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:24.272160 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:24.283807 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08a406-3db6-46ac-8098-155106fe6b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:24.286892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08a406-3db6-46ac-8098-155106fe6b94 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:24.286892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08a406-3db6-46ac-8098-155106fe6b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:24.286892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08a406-3db6-46ac-8098-155106fe6b94 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:24.286892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a08a406-3db6-46ac-8098-155106fe6b94 None admin] RBAC: Adding query filter params (name=image) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:24.286892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a08a406-3db6-46ac-8098-155106fe6b94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:24.287370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a08a406-3db6-46ac-8098-155106fe6b94 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:24.296909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 528/1049] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:31:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:24.297887 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09042fc-2e73-40d7-a401-e3c2965b4d58 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:24.377743 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:24.402077 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09042fc-2e73-40d7-a401-e3c2965b4d58 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:24.436350 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1013bd-be84-40fc-ad87-b06bdb7d1de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:24.436977 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1013bd-be84-40fc-ad87-b06bdb7d1de9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:24.437262 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1013bd-be84-40fc-ad87-b06bdb7d1de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:24.437543 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1013bd-be84-40fc-ad87-b06bdb7d1de9 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:24.438227 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1013bd-be84-40fc-ad87-b06bdb7d1de9 None admin] RBAC: Adding query filter params (type=image) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:24.438536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1013bd-be84-40fc-ad87-b06bdb7d1de9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:24.439937 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1013bd-be84-40fc-ad87-b06bdb7d1de9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:24.450287 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 522/1050] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:31:24 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:24.451442 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08a406-3db6-46ac-8098-155106fe6b94 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:24.554263 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08a406-3db6-46ac-8098-155106fe6b94 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:24.587560 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f647928-b81c-43fd-ab75-837bd27d9e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:24.588178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f647928-b81c-43fd-ab75-837bd27d9e32 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:24.588466 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f647928-b81c-43fd-ab75-837bd27d9e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:24.588746 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f647928-b81c-43fd-ab75-837bd27d9e32 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:24.589451 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f647928-b81c-43fd-ab75-837bd27d9e32 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:24.590870 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f647928-b81c-43fd-ab75-837bd27d9e32 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:24.669665 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 529/1051] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:31:24 2022] POST /identity/v3/endpoints => generated 327 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:31:25.401200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea2a1a-be97-4c72-b742-65c7555d6b26 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:25.401689 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea2a1a-be97-4c72-b742-65c7555d6b26 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:25.401689 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea2a1a-be97-4c72-b742-65c7555d6b26 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:25.403203 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 523/1052] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:25.408448 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3f62d-4c14-4c2c-a53a-a99f2ccc96f1 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:25.408742 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3f62d-4c14-4c2c-a53a-a99f2ccc96f1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:25.409022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3f62d-4c14-4c2c-a53a-a99f2ccc96f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:25.427451 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-80c3f62d-4c14-4c2c-a53a-a99f2ccc96f1 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:25.428636 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-80c3f62d-4c14-4c2c-a53a-a99f2ccc96f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:25.465262 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:25.503862 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 530/1053] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:25.526178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea2a1a-be97-4c72-b742-65c7555d6b26 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:25.629199 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea2a1a-be97-4c72-b742-65c7555d6b26 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:25.664218 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae3b02-8533-44d6-871c-ff0324e109b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:25.664786 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae3b02-8533-44d6-871c-ff0324e109b3 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:25.665070 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae3b02-8533-44d6-871c-ff0324e109b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:25.665344 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae3b02-8533-44d6-871c-ff0324e109b3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:25.673908 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:25.683263 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afae3b02-8533-44d6-871c-ff0324e109b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:31:25.688839 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-afae3b02-8533-44d6-871c-ff0324e109b3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 14:31:25.688839 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:25.688839 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:25.688839 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:25.688839 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:25.688839 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:25.688839 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:25.688839 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:25.688839 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:25.688839 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:25.688839 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:25.688839 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:25.688839 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:31:25.688839 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:31:25.688839 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:31:25.688839 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:31:25.689786 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:31:25.689786 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:31:25.689786 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:31:25.689786 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:31:25.689786 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:25.689786 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:25.689786 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:25.689786 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:25.689786 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:25.689786 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:25.689786 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:25.689786 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:25.689786 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:25.689786 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:25.689786 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:25.689786 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:25.690606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:25.690606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:25.690606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:25.690606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:25.690606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:31:25.690606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:31:25.690606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:31:25.690606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:31:25.690606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:31:25.690606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:31:25.690606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 14:31:25.690606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:25.690606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 524/1054] 10.222.0.97 () {56 vars in 1120 bytes} [Tue Jul 26 14:31:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:31:25.700715 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c3f62d-4c14-4c2c-a53a-a99f2ccc96f1 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:25.704861 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944583d-8d84-4305-9aa6-18ff03a122a7 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:25.705161 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944583d-8d84-4305-9aa6-18ff03a122a7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:25.705759 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944583d-8d84-4305-9aa6-18ff03a122a7 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:25.711444 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 525/1055] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:25.715197 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c4f1c-fb1a-472f-ab3f-a4dadfa9c2ba None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:25.715522 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c4f1c-fb1a-472f-ab3f-a4dadfa9c2ba None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:25.715801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c4f1c-fb1a-472f-ab3f-a4dadfa9c2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:25.735045 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c40c4f1c-fb1a-472f-ab3f-a4dadfa9c2ba None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:25.736261 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c40c4f1c-fb1a-472f-ab3f-a4dadfa9c2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:25.839211 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 526/1056] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:25.850248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c4f1c-fb1a-472f-ab3f-a4dadfa9c2ba None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:25.868672 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c3f62d-4c14-4c2c-a53a-a99f2ccc96f1 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:25.910368 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a1e24-b8ea-4068-af6d-7c8c127059e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:25.911323 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a1e24-b8ea-4068-af6d-7c8c127059e7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:25.911789 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a1e24-b8ea-4068-af6d-7c8c127059e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:25.912065 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a1e24-b8ea-4068-af6d-7c8c127059e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:25.912788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724a1e24-b8ea-4068-af6d-7c8c127059e7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:25.913095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724a1e24-b8ea-4068-af6d-7c8c127059e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:25.914949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724a1e24-b8ea-4068-af6d-7c8c127059e7 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:25.922944 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 531/1057] 10.222.0.97 () {56 vars in 1115 bytes} [Tue Jul 26 14:31:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:25.929184 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a1e24-b8ea-4068-af6d-7c8c127059e7 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:25.977649 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:25.981663 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c4f1c-fb1a-472f-ab3f-a4dadfa9c2ba None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:26.025996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:26.041140 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6e56a-d2d6-4646-a365-9c9770091060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:26.041976 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6e56a-d2d6-4646-a365-9c9770091060 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:26.042259 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6e56a-d2d6-4646-a365-9c9770091060 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:26.042534 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6e56a-d2d6-4646-a365-9c9770091060 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:26.070778 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6e56a-d2d6-4646-a365-9c9770091060 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:31:26.076898 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-37b6e56a-d2d6-4646-a365-9c9770091060 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:31:26.076898 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:26.076898 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:26.076898 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:26.076898 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:26.076898 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:26.076898 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:26.076898 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:26.076898 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:26.076898 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:26.076898 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:26.076898 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:26.076898 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:31:26.076898 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:31:26.076898 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:31:26.076898 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:31:26.077964 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:31:26.077964 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:31:26.077964 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:31:26.077964 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:31:26.077964 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:26.077964 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:26.077964 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:26.077964 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:26.077964 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:26.077964 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:26.077964 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:26.077964 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:26.077964 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:26.077964 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:26.077964 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:26.077964 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:26.078893 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:26.078893 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:26.078893 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:26.078893 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:26.078893 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:31:26.078893 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:31:26.078893 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:31:26.078893 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:31:26.078893 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:31:26.078893 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:31:26.078893 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:31:26.078893 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:26.083519 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 527/1058] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:31:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:31:26.083519 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6e56a-d2d6-4646-a365-9c9770091060 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:26.091044 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a1e24-b8ea-4068-af6d-7c8c127059e7 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:26.136864 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566da45-42e5-4489-9a13-09c8a93dcb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:26.137657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566da45-42e5-4489-9a13-09c8a93dcb1e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:26.138198 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566da45-42e5-4489-9a13-09c8a93dcb1e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:26.138586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566da45-42e5-4489-9a13-09c8a93dcb1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:26.149924 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a566da45-42e5-4489-9a13-09c8a93dcb1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:26.151341 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a566da45-42e5-4489-9a13-09c8a93dcb1e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:26.166047 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-a566da45-42e5-4489-9a13-09c8a93dcb1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:31:26.166047 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:26.166047 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:26.166047 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:26.166047 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:26.166047 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:26.166047 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:26.166047 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:26.166047 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:26.166047 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:26.166047 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:26.166047 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:26.166047 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:31:26.166047 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:31:26.166047 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:31:26.166926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:31:26.166926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:26.166926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:26.166926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:26.166926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:26.166926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:26.166926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:26.166926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:26.166926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:26.166926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:26.166926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:26.166926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:26.166926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:26.166926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:26.166926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:26.166926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:26.166926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:26.167858 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:31:26.167858 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:31:26.167858 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:31:26.167858 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:26.167858 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 532/1059] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:31:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:31:26.172783 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566da45-42e5-4489-9a13-09c8a93dcb1e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:26.197525 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6e56a-d2d6-4646-a365-9c9770091060 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:26.237764 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f901c0c-0342-42b0-b7e9-7665c67433b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:26.239127 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f901c0c-0342-42b0-b7e9-7665c67433b6 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:26.239631 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f901c0c-0342-42b0-b7e9-7665c67433b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:26.239911 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f901c0c-0342-42b0-b7e9-7665c67433b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:26.240624 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f901c0c-0342-42b0-b7e9-7665c67433b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:26.240931 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f901c0c-0342-42b0-b7e9-7665c67433b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:26.242294 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f901c0c-0342-42b0-b7e9-7665c67433b6 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:26.250932 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 528/1060] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:31:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:26.257369 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f901c0c-0342-42b0-b7e9-7665c67433b6 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:26.263945 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:26.283303 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566da45-42e5-4489-9a13-09c8a93dcb1e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:26.320557 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:26.322177 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8137ef-e354-4c86-8c2b-fb02c3300d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:26.322805 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8137ef-e354-4c86-8c2b-fb02c3300d6f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:26.323089 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8137ef-e354-4c86-8c2b-fb02c3300d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:26.323367 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8137ef-e354-4c86-8c2b-fb02c3300d6f None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:26.324086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8137ef-e354-4c86-8c2b-fb02c3300d6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:26.324401 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8137ef-e354-4c86-8c2b-fb02c3300d6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:26.325760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8137ef-e354-4c86-8c2b-fb02c3300d6f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:26.337534 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 533/1061] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:31:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:26.343368 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8137ef-e354-4c86-8c2b-fb02c3300d6f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:26.370396 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f901c0c-0342-42b0-b7e9-7665c67433b6 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:26.409200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c531158-3301-40d4-a810-940fa35301cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:26.410113 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c531158-3301-40d4-a810-940fa35301cb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:26.410431 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c531158-3301-40d4-a810-940fa35301cb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:26.410709 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c531158-3301-40d4-a810-940fa35301cb None admin] PATH_INFO: `/v3/users/ef69e9851acd4c60be511b1593c5a99a` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:26.425755 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c531158-3301-40d4-a810-940fa35301cb None admin] RBAC: Authorizing `identity:get_user(user_id=ef69e9851acd4c60be511b1593c5a99a)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:26.427139 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c531158-3301-40d4-a810-940fa35301cb None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:26.428877 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 529/1062] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:31:26 2022] GET /identity/v3/users/ef69e9851acd4c60be511b1593c5a99a => generated 285 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:26.434550 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c531158-3301-40d4-a810-940fa35301cb None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:26.451755 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8137ef-e354-4c86-8c2b-fb02c3300d6f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:26.487585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9c6d4-240d-4c2d-a2d7-1c0d0e0606f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:26.488781 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9c6d4-240d-4c2d-a2d7-1c0d0e0606f3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:26.489069 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9c6d4-240d-4c2d-a2d7-1c0d0e0606f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:26.489348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9c6d4-240d-4c2d-a2d7-1c0d0e0606f3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:26.497453 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de9c6d4-240d-4c2d-a2d7-1c0d0e0606f3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:26.498892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de9c6d4-240d-4c2d-a2d7-1c0d0e0606f3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:26.512591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-2de9c6d4-240d-4c2d-a2d7-1c0d0e0606f3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:31:26.512591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:26.512591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:26.512591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:26.512591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:26.512591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:26.512591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:26.512591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:26.512591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:26.512591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:26.512591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:26.512591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:26.512591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:31:26.512591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:31:26.512591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:31:26.512591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:31:26.514028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:26.514028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:26.514028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:31:26.514028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:31:26.514028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:31:26.514028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:31:26.514028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:26.514028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:26.514028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:26.514028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:26.514028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:26.514028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:26.514028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:26.514028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:26.514028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:26.514028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:26.514028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:26.514028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:26.514975 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:26.514975 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:26.514975 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:31:26.514975 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:31:26.514975 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:31:26.514975 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:31:26.514975 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:31:26.514975 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:31:26.514975 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:31:26.514975 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:26.514975 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 534/1063] 10.222.0.97 () {56 vars in 1093 bytes} [Tue Jul 26 14:31:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:31:26.519857 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9c6d4-240d-4c2d-a2d7-1c0d0e0606f3 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:26.522557 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:26.541998 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c531158-3301-40d4-a810-940fa35301cb None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:26.554380 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:26.578970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd666243-21d7-40af-b159-9ee48b8ac298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:26.579624 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd666243-21d7-40af-b159-9ee48b8ac298 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:26.579914 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd666243-21d7-40af-b159-9ee48b8ac298 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:26.580216 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd666243-21d7-40af-b159-9ee48b8ac298 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:26.626379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9c6d4-240d-4c2d-a2d7-1c0d0e0606f3 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:26.650617 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd666243-21d7-40af-b159-9ee48b8ac298 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:26.652101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd666243-21d7-40af-b159-9ee48b8ac298 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:26.662372 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60ef0d-1e9b-4a35-9afb-52422a64b678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:26.662995 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60ef0d-1e9b-4a35-9afb-52422a64b678 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:26.663419 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60ef0d-1e9b-4a35-9afb-52422a64b678 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:26.664998 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60ef0d-1e9b-4a35-9afb-52422a64b678 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:26.665801 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60ef0d-1e9b-4a35-9afb-52422a64b678 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:26.666134 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60ef0d-1e9b-4a35-9afb-52422a64b678 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:26.667778 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60ef0d-1e9b-4a35-9afb-52422a64b678 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:26.688145 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 535/1064] 10.222.0.97 () {56 vars in 1133 bytes} [Tue Jul 26 14:31:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:26.693843 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60ef0d-1e9b-4a35-9afb-52422a64b678 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:26.754403 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:26.760010 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 530/1065] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:31:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 14:31:26.761281 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d8a5f-01ec-401f-bc75-61e922d76dc9 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:26.761641 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d8a5f-01ec-401f-bc75-61e922d76dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:26.761926 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d8a5f-01ec-401f-bc75-61e922d76dc9 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:26.769683 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 531/1066] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:26.769683 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d8a5f-01ec-401f-bc75-61e922d76dc9 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:26.778839 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:26.795276 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60ef0d-1e9b-4a35-9afb-52422a64b678 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:26.829433 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c100336-6d32-4ad0-9620-4e71c57af9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:26.830064 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c100336-6d32-4ad0-9620-4e71c57af9c7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:26.830356 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c100336-6d32-4ad0-9620-4e71c57af9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:26.830634 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c100336-6d32-4ad0-9620-4e71c57af9c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:26.841346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c100336-6d32-4ad0-9620-4e71c57af9c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:26.842789 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c100336-6d32-4ad0-9620-4e71c57af9c7 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:26.858954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-3c100336-6d32-4ad0-9620-4e71c57af9c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:31:26.858954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:26.858954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:26.858954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:26.858954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:26.858954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:26.858954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:26.858954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:26.858954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:26.858954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:26.858954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:26.858954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:26.858954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:31:26.858954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:31:26.858954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:31:26.859868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:31:26.859868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:26.859868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:26.859868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:26.859868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:26.859868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:26.859868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:26.859868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:26.859868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:26.859868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:26.859868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:26.859868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:26.859868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:26.859868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:26.859868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:26.859868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:26.859868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:26.860772 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:31:26.860772 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:31:26.860772 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:31:26.860772 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:26.860772 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 536/1067] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:31:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:31:26.861348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c12f2d-db4d-4416-ae89-788b9dc0604f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:26.862399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c12f2d-db4d-4416-ae89-788b9dc0604f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:26.862399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c12f2d-db4d-4416-ae89-788b9dc0604f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:26.868651 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d8a5f-01ec-401f-bc75-61e922d76dc9 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:26.879510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d5c12f2d-db4d-4416-ae89-788b9dc0604f None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:26.880714 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d5c12f2d-db4d-4416-ae89-788b9dc0604f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:26.903110 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fafae5-6e38-414e-9999-a638a910a25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:26.903782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fafae5-6e38-414e-9999-a638a910a25f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:26.904068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fafae5-6e38-414e-9999-a638a910a25f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:26.904342 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fafae5-6e38-414e-9999-a638a910a25f None admin] PATH_INFO: `/v3/projects/bba64eea79c64df19fe3e59b9d745d1c` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:26.905485 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fafae5-6e38-414e-9999-a638a910a25f None admin] RBAC: Authorizing `identity:get_project(project_id=bba64eea79c64df19fe3e59b9d745d1c)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:26.906905 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fafae5-6e38-414e-9999-a638a910a25f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:26.908600 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 532/1068] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:31:26 2022] GET /identity/v3/projects/bba64eea79c64df19fe3e59b9d745d1c => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:26.912259 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fafae5-6e38-414e-9999-a638a910a25f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:26.946868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 537/1069] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:26.947660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c12f2d-db4d-4416-ae89-788b9dc0604f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:26.990513 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:26.997050 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:27.011958 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fafae5-6e38-414e-9999-a638a910a25f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:27.048236 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1dcab7-cef6-4c60-b7d8-443c16dee8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:27.049643 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1dcab7-cef6-4c60-b7d8-443c16dee8b2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:27.050879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1dcab7-cef6-4c60-b7d8-443c16dee8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:27.051314 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1dcab7-cef6-4c60-b7d8-443c16dee8b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:27.052558 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1dcab7-cef6-4c60-b7d8-443c16dee8b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:27.053041 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1dcab7-cef6-4c60-b7d8-443c16dee8b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:27.055255 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c12f2d-db4d-4416-ae89-788b9dc0604f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:27.055395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1dcab7-cef6-4c60-b7d8-443c16dee8b2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:27.069011 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 533/1070] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:31:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:27.069919 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1dcab7-cef6-4c60-b7d8-443c16dee8b2 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:27.090217 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b14f56-59b2-466a-b3b7-c20d698ab8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:27.090829 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b14f56-59b2-466a-b3b7-c20d698ab8ba None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:27.091113 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b14f56-59b2-466a-b3b7-c20d698ab8ba None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:27.091388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b14f56-59b2-466a-b3b7-c20d698ab8ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:27.092201 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b14f56-59b2-466a-b3b7-c20d698ab8ba None admin] RBAC: Adding query filter params (role.id=51b2d10dfffb4b75b186f3d8b15e2a78, scope.project.id=bba64eea79c64df19fe3e59b9d745d1c, user.id=ef69e9851acd4c60be511b1593c5a99a) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:27.092511 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b14f56-59b2-466a-b3b7-c20d698ab8ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:27.093933 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b14f56-59b2-466a-b3b7-c20d698ab8ba None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:27.099882 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 538/1071] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:31:26 2022] GET /identity/v3/role_assignments?user.id=ef69e9851acd4c60be511b1593c5a99a&scope.project.id=bba64eea79c64df19fe3e59b9d745d1c&role.id=51b2d10dfffb4b75b186f3d8b15e2a78 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:27.100812 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b14f56-59b2-466a-b3b7-c20d698ab8ba None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:27.191090 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1dcab7-cef6-4c60-b7d8-443c16dee8b2 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:27.219094 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b14f56-59b2-466a-b3b7-c20d698ab8ba None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:27.228747 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:27.230790 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee75762-4eba-4978-b3bc-a0988b375c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:27.231223 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee75762-4eba-4978-b3bc-a0988b375c26 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:27.231533 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee75762-4eba-4978-b3bc-a0988b375c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:27.231828 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee75762-4eba-4978-b3bc-a0988b375c26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:27.242961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:27.259878 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a530e05-8331-4402-a8d5-7b089945eee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:27.260098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a530e05-8331-4402-a8d5-7b089945eee1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:27.260382 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a530e05-8331-4402-a8d5-7b089945eee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:27.260658 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a530e05-8331-4402-a8d5-7b089945eee1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:27.267986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a530e05-8331-4402-a8d5-7b089945eee1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:27.269977 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a530e05-8331-4402-a8d5-7b089945eee1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:27.281821 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-4a530e05-8331-4402-a8d5-7b089945eee1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:31:27.281821 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:27.281821 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:27.281821 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:27.281821 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:27.281821 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:27.281821 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:27.281821 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:27.281821 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:27.281821 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:27.281821 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:27.281821 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:27.281821 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:31:27.281821 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:31:27.281821 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:31:27.291012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:31:27.291012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:27.291012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:27.291012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:27.291012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:27.291012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:27.291012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:27.291012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:27.291012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:27.291012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:27.291012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:27.291012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:27.291012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:27.291012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:27.291012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:27.291012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:27.291012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:27.292160 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:31:27.292160 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:31:27.292160 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:31:27.292160 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:31:27.292160 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:31:27.292160 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:31:27.292160 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:31:27.292160 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:27.292160 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 539/1072] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:31:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:31:27.292160 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a530e05-8331-4402-a8d5-7b089945eee1 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:27.312768 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee75762-4eba-4978-b3bc-a0988b375c26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:27.314522 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee75762-4eba-4978-b3bc-a0988b375c26 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:27.396420 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a530e05-8331-4402-a8d5-7b089945eee1 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:27.425008 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 534/1073] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:31:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:31:27.431189 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee75762-4eba-4978-b3bc-a0988b375c26 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:27.438837 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e107e3-7c5e-4a12-9fff-349f52c78aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:27.440035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e107e3-7c5e-4a12-9fff-349f52c78aee None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:27.440325 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e107e3-7c5e-4a12-9fff-349f52c78aee None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:27.440604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e107e3-7c5e-4a12-9fff-349f52c78aee None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:27.441475 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e107e3-7c5e-4a12-9fff-349f52c78aee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:27.442158 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e107e3-7c5e-4a12-9fff-349f52c78aee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:27.444243 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e107e3-7c5e-4a12-9fff-349f52c78aee None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:27.463029 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 540/1074] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:31:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:27.464580 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e107e3-7c5e-4a12-9fff-349f52c78aee None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:27.481797 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:27.495266 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:27.549369 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee75762-4eba-4978-b3bc-a0988b375c26 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:27.573580 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e107e3-7c5e-4a12-9fff-349f52c78aee None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:27.589750 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61535655-6c4e-472c-8f36-53d6aa7fc9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:27.590404 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61535655-6c4e-472c-8f36-53d6aa7fc9a5 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:27.590691 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61535655-6c4e-472c-8f36-53d6aa7fc9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:27.590965 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61535655-6c4e-472c-8f36-53d6aa7fc9a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:27.602366 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61535655-6c4e-472c-8f36-53d6aa7fc9a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:27.603858 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61535655-6c4e-472c-8f36-53d6aa7fc9a5 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:27.611260 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09008300-8800-457b-800e-50ce85a30707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:27.611963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09008300-8800-457b-800e-50ce85a30707 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:27.612244 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09008300-8800-457b-800e-50ce85a30707 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:27.612544 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09008300-8800-457b-800e-50ce85a30707 None admin] PATH_INFO: `/v3/projects/c4c8da3011294783b2180ade1bfd9366/users/266abef6cda5495b897170f92043aa03/roles/a61134ee45dd49b3a20a8a55044e0e3c` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:27.615349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09008300-8800-457b-800e-50ce85a30707 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4c8da3011294783b2180ade1bfd9366, user_id=266abef6cda5495b897170f92043aa03, role_id=a61134ee45dd49b3a20a8a55044e0e3c)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:27.616802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09008300-8800-457b-800e-50ce85a30707 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:27.620495 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-61535655-6c4e-472c-8f36-53d6aa7fc9a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:31:27.620495 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:27.620495 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:27.620495 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:27.620495 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:27.620495 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:27.620495 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:27.620495 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:27.620495 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:27.620495 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:27.620495 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:27.620495 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:27.620495 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:31:27.620495 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:31:27.620495 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:31:27.621364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:31:27.621364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:27.621364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:27.621364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:27.621364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:27.621364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:27.621364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:27.621364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:27.621364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:27.621364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:27.621364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:27.621364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:27.621364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:27.621364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:27.621364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:27.621364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:27.621364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:27.622243 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:31:27.622243 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:31:27.622243 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:31:27.622243 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:27.622243 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 535/1075] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:31:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:31:27.627329 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61535655-6c4e-472c-8f36-53d6aa7fc9a5 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:27.718053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:27.728200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61535655-6c4e-472c-8f36-53d6aa7fc9a5 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:27.751956 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 541/1076] 10.222.0.97 () {58 vars in 1439 bytes} [Tue Jul 26 14:31:27 2022] PUT /identity/v3/projects/c4c8da3011294783b2180ade1bfd9366/users/266abef6cda5495b897170f92043aa03/roles/a61134ee45dd49b3a20a8a55044e0e3c => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:31:27.762465 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d36e8cc-e50a-4cdc-ac2a-00e9f1e201a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:27.763563 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36e8cc-e50a-4cdc-ac2a-00e9f1e201a2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:27.763782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36e8cc-e50a-4cdc-ac2a-00e9f1e201a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:27.764129 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36e8cc-e50a-4cdc-ac2a-00e9f1e201a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:27.764848 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d36e8cc-e50a-4cdc-ac2a-00e9f1e201a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:27.765201 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d36e8cc-e50a-4cdc-ac2a-00e9f1e201a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:27.766756 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d36e8cc-e50a-4cdc-ac2a-00e9f1e201a2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:27.786768 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 536/1077] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:31:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:29.086830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab5272-3e5e-4ce5-9341-45eac1a9d4b6 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:29.087277 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab5272-3e5e-4ce5-9341-45eac1a9d4b6 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:29.087373 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab5272-3e5e-4ce5-9341-45eac1a9d4b6 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:29.088968 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 542/1078] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:29.094092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84baad9f-fdb7-48d2-878f-688f54dc59ec None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:29.094483 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84baad9f-fdb7-48d2-878f-688f54dc59ec None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:29.094648 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84baad9f-fdb7-48d2-878f-688f54dc59ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:29.113543 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-84baad9f-fdb7-48d2-878f-688f54dc59ec None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:29.114733 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-84baad9f-fdb7-48d2-878f-688f54dc59ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:29.181388 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 537/1079] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:29.204128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab5272-3e5e-4ce5-9341-45eac1a9d4b6 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:29.290022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:29.305610 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab5272-3e5e-4ce5-9341-45eac1a9d4b6 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:29.338990 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1659622-ca1c-43b1-896b-d935ea882d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:29.339762 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1659622-ca1c-43b1-896b-d935ea882d3f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:29.340049 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1659622-ca1c-43b1-896b-d935ea882d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:29.340329 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1659622-ca1c-43b1-896b-d935ea882d3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:29.351392 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1659622-ca1c-43b1-896b-d935ea882d3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:31:29.356349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-d1659622-ca1c-43b1-896b-d935ea882d3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:31:29.356349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:29.356349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:29.356349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:29.356349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:29.356349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:29.356349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:29.356349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:29.356349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:29.356349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:29.356349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:29.356349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:29.356349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:31:29.356349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:31:29.356349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:31:29.356349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:31:29.357242 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:31:29.357242 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:31:29.357242 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:31:29.357242 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:31:29.357242 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:29.357242 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:29.357242 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:29.357242 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:29.357242 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:29.357242 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:29.357242 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:29.357242 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:29.357242 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:29.357242 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:29.357242 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:29.357242 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:29.358082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:29.358082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:29.358082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:29.358082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:29.358082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:31:29.358082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:31:29.358082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:31:29.358082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:31:29.358082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:31:29.358082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:31:29.358082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:31:29.358082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:29.359692 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 543/1080] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:31:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:31:29.364287 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84baad9f-fdb7-48d2-878f-688f54dc59ec None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:29.442286 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:29.461225 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84baad9f-fdb7-48d2-878f-688f54dc59ec None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:29.495136 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b69e16-093d-47db-880d-4448da7f229d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:29.495779 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b69e16-093d-47db-880d-4448da7f229d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:29.496062 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b69e16-093d-47db-880d-4448da7f229d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:29.496334 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b69e16-093d-47db-880d-4448da7f229d None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:29.497041 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b69e16-093d-47db-880d-4448da7f229d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:29.497342 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b69e16-093d-47db-880d-4448da7f229d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:29.498760 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b69e16-093d-47db-880d-4448da7f229d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:29.506781 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 538/1081] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:31:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:29.512493 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1659622-ca1c-43b1-896b-d935ea882d3f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:29.611454 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1659622-ca1c-43b1-896b-d935ea882d3f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:29.645099 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9d5ae-85bd-4b76-85c4-2946ada7e0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:29.645735 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9d5ae-85bd-4b76-85c4-2946ada7e0fd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:29.646017 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9d5ae-85bd-4b76-85c4-2946ada7e0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:29.646293 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9d5ae-85bd-4b76-85c4-2946ada7e0fd None admin] PATH_INFO: `/v3/users/ef69e9851acd4c60be511b1593c5a99a` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:29.647473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d9d5ae-85bd-4b76-85c4-2946ada7e0fd None admin] RBAC: Authorizing `identity:get_user(user_id=ef69e9851acd4c60be511b1593c5a99a)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:29.648901 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d9d5ae-85bd-4b76-85c4-2946ada7e0fd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:29.650486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 544/1082] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:31:29 2022] GET /identity/v3/users/ef69e9851acd4c60be511b1593c5a99a => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:29.656246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b69e16-093d-47db-880d-4448da7f229d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:29.755026 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b69e16-093d-47db-880d-4448da7f229d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:29.767904 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773061d3-b03f-45dd-ab29-b63d578e7017 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:29.768196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773061d3-b03f-45dd-ab29-b63d578e7017 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:29.768476 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773061d3-b03f-45dd-ab29-b63d578e7017 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:29.769921 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 545/1083] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:31:29.774651 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7121ae-1669-47f3-8777-6ac324a4d8d8 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:29.774945 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7121ae-1669-47f3-8777-6ac324a4d8d8 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:29.775255 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7121ae-1669-47f3-8777-6ac324a4d8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:29.788467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a3739-8ae9-479b-a879-9cfe0c92053d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:29.789102 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a3739-8ae9-479b-a879-9cfe0c92053d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:29.789381 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a3739-8ae9-479b-a879-9cfe0c92053d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:29.789652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a3739-8ae9-479b-a879-9cfe0c92053d None admin] PATH_INFO: `/v3/projects/bba64eea79c64df19fe3e59b9d745d1c` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:29.790776 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a3739-8ae9-479b-a879-9cfe0c92053d None admin] RBAC: Authorizing `identity:get_project(project_id=bba64eea79c64df19fe3e59b9d745d1c)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:29.792219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a3739-8ae9-479b-a879-9cfe0c92053d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:29.793287 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0c7121ae-1669-47f3-8777-6ac324a4d8d8 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:29.793877 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 539/1084] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:31:29 2022] GET /identity/v3/projects/bba64eea79c64df19fe3e59b9d745d1c => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:29.794486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0c7121ae-1669-47f3-8777-6ac324a4d8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:29.799871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a3739-8ae9-479b-a879-9cfe0c92053d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:29.804530 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:29.812462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:29.860374 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 546/1085] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 14:31:29.883915 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7121ae-1669-47f3-8777-6ac324a4d8d8 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:29.900402 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a3739-8ae9-479b-a879-9cfe0c92053d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:29.933988 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf88ed7-f75c-4f44-b9f9-9e786bbd583f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:29.934649 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf88ed7-f75c-4f44-b9f9-9e786bbd583f None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:29.934939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf88ed7-f75c-4f44-b9f9-9e786bbd583f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:29.935218 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf88ed7-f75c-4f44-b9f9-9e786bbd583f None admin] PATH_INFO: `/v3/projects/bba64eea79c64df19fe3e59b9d745d1c/users/ef69e9851acd4c60be511b1593c5a99a/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:29.937156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf88ed7-f75c-4f44-b9f9-9e786bbd583f None admin] RBAC: Authorizing `identity:create_grant(project_id=bba64eea79c64df19fe3e59b9d745d1c, user_id=ef69e9851acd4c60be511b1593c5a99a, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:29.938575 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf88ed7-f75c-4f44-b9f9-9e786bbd583f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:29.979846 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7121ae-1669-47f3-8777-6ac324a4d8d8 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:30.013408 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155d9139-8629-46fb-97cb-6c5f0dc848d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:30.014046 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d9139-8629-46fb-97cb-6c5f0dc848d6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:30.014324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d9139-8629-46fb-97cb-6c5f0dc848d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:30.014600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d9139-8629-46fb-97cb-6c5f0dc848d6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:30.019556 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 540/1086] 10.222.0.97 () {58 vars in 1439 bytes} [Tue Jul 26 14:31:29 2022] PUT /identity/v3/projects/bba64eea79c64df19fe3e59b9d745d1c/users/ef69e9851acd4c60be511b1593c5a99a/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:31:30.025212 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155d9139-8629-46fb-97cb-6c5f0dc848d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:31:30.030305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-155d9139-8629-46fb-97cb-6c5f0dc848d6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 14:31:30.030305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:30.030305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:30.030305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:30.030305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:30.030305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:30.030305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:30.030305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:30.030305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:30.030305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:30.030305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:30.030305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:30.030305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:31:30.030305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:31:30.030305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:31:30.030305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:31:30.033655 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:31:30.033655 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:31:30.033655 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:31:30.033655 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:31:30.033655 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:30.033655 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:30.033655 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:30.033655 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:30.033655 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:30.033655 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:30.033655 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:30.033655 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:30.033655 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:30.033655 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:30.033655 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:30.033655 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:30.035348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:30.035348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:30.035348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:30.035348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:30.035348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:31:30.035348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:31:30.035348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:31:30.035348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:31:30.035348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:31:30.035348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:31:30.035348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 14:31:30.035348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:30.035348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 547/1087] 10.222.0.97 () {56 vars in 1120 bytes} [Tue Jul 26 14:31:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:31:30.038583 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf88ed7-f75c-4f44-b9f9-9e786bbd583f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:30.117060 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:30.149925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf88ed7-f75c-4f44-b9f9-9e786bbd583f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:30.187548 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e778a8-ac1a-4643-9aca-cf3531ae305f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:30.188219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e778a8-ac1a-4643-9aca-cf3531ae305f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:30.188499 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e778a8-ac1a-4643-9aca-cf3531ae305f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:30.188775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e778a8-ac1a-4643-9aca-cf3531ae305f None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:30.189491 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e778a8-ac1a-4643-9aca-cf3531ae305f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:30.189803 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e778a8-ac1a-4643-9aca-cf3531ae305f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:30.191249 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e778a8-ac1a-4643-9aca-cf3531ae305f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:30.199737 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 541/1088] 10.222.0.97 () {56 vars in 1115 bytes} [Tue Jul 26 14:31:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:30.205596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155d9139-8629-46fb-97cb-6c5f0dc848d6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:30.211219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:30.308780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155d9139-8629-46fb-97cb-6c5f0dc848d6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:30.341701 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad0b6c3-d4aa-4a14-9e09-a74d5333652d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:30.342340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad0b6c3-d4aa-4a14-9e09-a74d5333652d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:30.342623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad0b6c3-d4aa-4a14-9e09-a74d5333652d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:30.342899 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad0b6c3-d4aa-4a14-9e09-a74d5333652d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:30.353312 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad0b6c3-d4aa-4a14-9e09-a74d5333652d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:30.354735 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad0b6c3-d4aa-4a14-9e09-a74d5333652d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:30.369165 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-fad0b6c3-d4aa-4a14-9e09-a74d5333652d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:31:30.369165 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:30.369165 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:30.369165 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:30.369165 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:30.369165 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:30.369165 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:30.369165 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:30.369165 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:30.369165 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:30.369165 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:30.369165 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:30.369165 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:31:30.369165 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:31:30.369165 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:31:30.370223 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:31:30.370223 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:30.370223 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:30.370223 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:30.370223 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:30.370223 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:30.370223 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:30.370223 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:30.370223 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:30.370223 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:30.370223 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:30.370223 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:30.370223 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:30.370223 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:30.370223 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:30.370223 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:30.370223 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:30.375819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:31:30.375819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:31:30.375819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:31:30.375819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:30.375819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 548/1089] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:31:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:31:30.376119 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e778a8-ac1a-4643-9aca-cf3531ae305f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:30.474050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e778a8-ac1a-4643-9aca-cf3531ae305f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:30.504444 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:30.509414 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9404823f-a6c2-48f7-ab07-e3efb71b5c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:30.510040 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404823f-a6c2-48f7-ab07-e3efb71b5c35 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:30.510318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404823f-a6c2-48f7-ab07-e3efb71b5c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:30.510589 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404823f-a6c2-48f7-ab07-e3efb71b5c35 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:30.511271 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9404823f-a6c2-48f7-ab07-e3efb71b5c35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:30.511610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9404823f-a6c2-48f7-ab07-e3efb71b5c35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:30.513280 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9404823f-a6c2-48f7-ab07-e3efb71b5c35 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:30.524845 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 542/1090] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:31:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:30.530348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad0b6c3-d4aa-4a14-9e09-a74d5333652d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:30.586067 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:30.630735 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad0b6c3-d4aa-4a14-9e09-a74d5333652d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:30.664620 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56a3c5-a961-4d44-bf37-68518e0489f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:30.665251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56a3c5-a961-4d44-bf37-68518e0489f2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:30.665532 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56a3c5-a961-4d44-bf37-68518e0489f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:30.665806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56a3c5-a961-4d44-bf37-68518e0489f2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:30.673323 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56a3c5-a961-4d44-bf37-68518e0489f2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:30.674766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56a3c5-a961-4d44-bf37-68518e0489f2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:30.686477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-8a56a3c5-a961-4d44-bf37-68518e0489f2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:31:30.686477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:30.686477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:30.686477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:30.686477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:30.686477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:30.686477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:30.686477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:30.686477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:30.686477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:30.686477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:30.686477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:30.686477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:31:30.686477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:31:30.686477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:31:30.686477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:31:30.687596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:30.687596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:30.687596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:31:30.687596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:31:30.687596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:31:30.687596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:31:30.687596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:30.687596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:30.687596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:30.687596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:30.687596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:30.687596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:30.687596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:30.687596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:30.687596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:30.687596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:30.687596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:30.687596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:30.688708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:30.688708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:30.688708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:31:30.688708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:31:30.688708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:31:30.688708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:31:30.688708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:31:30.688708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:31:30.688708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:31:30.688708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:30.691681 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 549/1091] 10.222.0.97 () {56 vars in 1093 bytes} [Tue Jul 26 14:31:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:31:30.693416 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9404823f-a6c2-48f7-ab07-e3efb71b5c35 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:30.794452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9404823f-a6c2-48f7-ab07-e3efb71b5c35 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:30.827981 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b053e8d-95ca-4445-b977-f7a1a0b35f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:30.828595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b053e8d-95ca-4445-b977-f7a1a0b35f08 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:30.828881 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b053e8d-95ca-4445-b977-f7a1a0b35f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:30.829160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b053e8d-95ca-4445-b977-f7a1a0b35f08 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:30.830012 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b053e8d-95ca-4445-b977-f7a1a0b35f08 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:30.830320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b053e8d-95ca-4445-b977-f7a1a0b35f08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:30.831789 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b053e8d-95ca-4445-b977-f7a1a0b35f08 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:30.849242 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 543/1092] 10.222.0.97 () {56 vars in 1133 bytes} [Tue Jul 26 14:31:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:30.854815 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56a3c5-a961-4d44-bf37-68518e0489f2 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:30.952780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56a3c5-a961-4d44-bf37-68518e0489f2 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:30.968887 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:30.986739 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1db6b-760d-4d32-8e31-93dcf5a54add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:30.987361 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1db6b-760d-4d32-8e31-93dcf5a54add None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:30.987669 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1db6b-760d-4d32-8e31-93dcf5a54add None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:30.987946 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1db6b-760d-4d32-8e31-93dcf5a54add None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:31.057697 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e1db6b-760d-4d32-8e31-93dcf5a54add None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:31.059161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e1db6b-760d-4d32-8e31-93dcf5a54add None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:31.156879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 550/1093] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:31:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:31:31.162560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b053e8d-95ca-4445-b977-f7a1a0b35f08 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:31.213236 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:31.263413 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b053e8d-95ca-4445-b977-f7a1a0b35f08 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:31.299001 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a7d22-e483-4171-ab37-1c54b38fef58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:31.300318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a7d22-e483-4171-ab37-1c54b38fef58 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:31.300613 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a7d22-e483-4171-ab37-1c54b38fef58 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:31.300892 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a7d22-e483-4171-ab37-1c54b38fef58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:31.310933 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4a7d22-e483-4171-ab37-1c54b38fef58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:31.312332 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4a7d22-e483-4171-ab37-1c54b38fef58 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:31.327027 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-bb4a7d22-e483-4171-ab37-1c54b38fef58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:31:31.327027 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:31.327027 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:31.327027 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:31.327027 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:31.327027 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:31.327027 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:31.327027 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:31.327027 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:31.327027 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:31.327027 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:31.327027 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:31.327027 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:31:31.327027 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:31:31.327027 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:31:31.327988 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:31:31.327988 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:31.327988 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:31.327988 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:31.327988 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:31.327988 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:31.327988 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:31.327988 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:31.327988 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:31.327988 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:31.327988 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:31.327988 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:31.327988 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:31.327988 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:31.327988 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:31.327988 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:31.327988 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:31.328867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:31:31.328867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:31:31.328867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:31:31.328867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:31.328867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 544/1094] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:31:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:31:31.334399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1db6b-760d-4d32-8e31-93dcf5a54add None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:31.351382 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:31.432032 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1db6b-760d-4d32-8e31-93dcf5a54add None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:31.465685 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db8471-f694-4d61-a5e1-aeda55367e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:31.466301 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db8471-f694-4d61-a5e1-aeda55367e3a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:31.466588 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db8471-f694-4d61-a5e1-aeda55367e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:31.466864 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db8471-f694-4d61-a5e1-aeda55367e3a None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:31.467581 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9db8471-f694-4d61-a5e1-aeda55367e3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:31.467888 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9db8471-f694-4d61-a5e1-aeda55367e3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:31.469238 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9db8471-f694-4d61-a5e1-aeda55367e3a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:31.480962 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 551/1095] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:31:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:31.486469 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a7d22-e483-4171-ab37-1c54b38fef58 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:31.586774 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a7d22-e483-4171-ab37-1c54b38fef58 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:31.588292 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:31.621757 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0398d-6ed3-473d-aa6a-9b842c87b626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:31.622395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0398d-6ed3-473d-aa6a-9b842c87b626 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:31.622677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0398d-6ed3-473d-aa6a-9b842c87b626 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:31.622955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0398d-6ed3-473d-aa6a-9b842c87b626 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:31.630441 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f0398d-6ed3-473d-aa6a-9b842c87b626 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:31.631922 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f0398d-6ed3-473d-aa6a-9b842c87b626 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:31.643961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-44f0398d-6ed3-473d-aa6a-9b842c87b626 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:31:31.643961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:31.643961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:31.643961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:31.643961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:31.643961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:31.643961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:31.643961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:31.643961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:31.643961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:31.643961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:31.643961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:31.643961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:31:31.643961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:31:31.643961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:31:31.645036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:31:31.645036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:31.645036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:31.645036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:31.645036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:31.645036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:31.645036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:31.645036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:31.645036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:31.645036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:31.645036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:31.645036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:31.645036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:31.645036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:31.645036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:31.645036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:31.645036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:31.650657 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:31:31.650657 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:31:31.650657 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:31:31.650657 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:31:31.650657 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:31:31.650657 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:31:31.650657 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:31:31.650657 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:31.650657 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 545/1096] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:31:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:31:31.651161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db8471-f694-4d61-a5e1-aeda55367e3a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:31.726661 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:31.748642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db8471-f694-4d61-a5e1-aeda55367e3a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:31.782017 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5180669-a10d-40db-b69c-87a51d5e2eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:31.782636 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5180669-a10d-40db-b69c-87a51d5e2eaf None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:31.782922 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5180669-a10d-40db-b69c-87a51d5e2eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:31.783200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5180669-a10d-40db-b69c-87a51d5e2eaf None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:31.783987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5180669-a10d-40db-b69c-87a51d5e2eaf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:31.784305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5180669-a10d-40db-b69c-87a51d5e2eaf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:31.785735 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5180669-a10d-40db-b69c-87a51d5e2eaf None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:31.797649 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 552/1097] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:31:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:31:31.804251 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0398d-6ed3-473d-aa6a-9b842c87b626 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:31.904766 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0398d-6ed3-473d-aa6a-9b842c87b626 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:31.937507 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50c584-cbed-41a6-b78a-42b04a72f172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:31.938116 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50c584-cbed-41a6-b78a-42b04a72f172 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:31.938395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50c584-cbed-41a6-b78a-42b04a72f172 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:31.938666 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50c584-cbed-41a6-b78a-42b04a72f172 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:31.939534 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff50c584-cbed-41a6-b78a-42b04a72f172 None admin] RBAC: Adding query filter params (role.id=a61134ee45dd49b3a20a8a55044e0e3c, scope.project.id=c4c8da3011294783b2180ade1bfd9366, user.id=266abef6cda5495b897170f92043aa03) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:31.939850 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff50c584-cbed-41a6-b78a-42b04a72f172 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:31.941423 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff50c584-cbed-41a6-b78a-42b04a72f172 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:31.947107 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 546/1098] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:31:31 2022] GET /identity/v3/role_assignments?user.id=266abef6cda5495b897170f92043aa03&scope.project.id=c4c8da3011294783b2180ade1bfd9366&role.id=a61134ee45dd49b3a20a8a55044e0e3c => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:32.030440 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab9cb0-fb0b-4a03-9723-1dfbc8ecb960 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:32.030725 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab9cb0-fb0b-4a03-9723-1dfbc8ecb960 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:32.030996 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab9cb0-fb0b-4a03-9723-1dfbc8ecb960 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:32.032539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 553/1099] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:32.040138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36418cac-0a03-487c-bbfc-8f63b41e849d None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:32.040138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36418cac-0a03-487c-bbfc-8f63b41e849d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:32.040138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36418cac-0a03-487c-bbfc-8f63b41e849d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:32.058169 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-36418cac-0a03-487c-bbfc-8f63b41e849d None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:32.059389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-36418cac-0a03-487c-bbfc-8f63b41e849d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:32.082113 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:32.131890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 547/1100] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:32.156088 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab9cb0-fb0b-4a03-9723-1dfbc8ecb960 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:32.261049 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab9cb0-fb0b-4a03-9723-1dfbc8ecb960 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:32.298123 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6143d-66df-4fcf-94d1-3cf7a4f5a406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:32.298768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6143d-66df-4fcf-94d1-3cf7a4f5a406 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:32.299061 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6143d-66df-4fcf-94d1-3cf7a4f5a406 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:32.299339 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6143d-66df-4fcf-94d1-3cf7a4f5a406 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:32.302123 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:32.310248 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d6143d-66df-4fcf-94d1-3cf7a4f5a406 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:31:32.315980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-a9d6143d-66df-4fcf-94d1-3cf7a4f5a406 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:31:32.315980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:32.315980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:32.315980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:32.315980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:32.315980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:32.315980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:32.315980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:32.315980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:32.315980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:32.315980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:32.315980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:32.315980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:31:32.315980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:31:32.315980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:31:32.315980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:31:32.317246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:31:32.317246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:31:32.317246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:31:32.317246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:31:32.317246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:32.317246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:32.317246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:32.317246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:32.317246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:32.317246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:32.317246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:32.317246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:32.317246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:32.317246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:32.317246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:32.317246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:32.318156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:32.318156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:32.318156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:32.318156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:32.318156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:31:32.318156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:31:32.318156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:31:32.318156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:31:32.318156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:31:32.318156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:31:32.318156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:31:32.318156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:32.318156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 554/1101] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:31:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:31:32.323792 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36418cac-0a03-487c-bbfc-8f63b41e849d None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:32.423973 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36418cac-0a03-487c-bbfc-8f63b41e849d None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:32.458422 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c99a25a-b182-40d1-bdb0-71c9ae643214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:32.459049 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99a25a-b182-40d1-bdb0-71c9ae643214 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:32.459333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99a25a-b182-40d1-bdb0-71c9ae643214 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:32.459645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99a25a-b182-40d1-bdb0-71c9ae643214 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:32.460363 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c99a25a-b182-40d1-bdb0-71c9ae643214 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:32.460670 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c99a25a-b182-40d1-bdb0-71c9ae643214 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:32.462036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c99a25a-b182-40d1-bdb0-71c9ae643214 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:32.470312 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 548/1102] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:31:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:32.476339 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6143d-66df-4fcf-94d1-3cf7a4f5a406 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:32.581068 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6143d-66df-4fcf-94d1-3cf7a4f5a406 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:32.615345 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7cb4b-f2dd-429d-8d79-2247eb07ca39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:32.616048 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7cb4b-f2dd-429d-8d79-2247eb07ca39 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:32.616335 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7cb4b-f2dd-429d-8d79-2247eb07ca39 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:32.616613 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7cb4b-f2dd-429d-8d79-2247eb07ca39 None admin] PATH_INFO: `/v3/users/ef69e9851acd4c60be511b1593c5a99a` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:32.617773 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7cb4b-f2dd-429d-8d79-2247eb07ca39 None admin] RBAC: Authorizing `identity:get_user(user_id=ef69e9851acd4c60be511b1593c5a99a)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:32.619196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7cb4b-f2dd-429d-8d79-2247eb07ca39 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:32.620812 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 555/1103] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:31:32 2022] GET /identity/v3/users/ef69e9851acd4c60be511b1593c5a99a => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:32.626713 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c99a25a-b182-40d1-bdb0-71c9ae643214 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:32.647374 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:32.731780 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c99a25a-b182-40d1-bdb0-71c9ae643214 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:32.767955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c6eb7-be91-4efd-85b6-e835bb7aebb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:32.768620 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c6eb7-be91-4efd-85b6-e835bb7aebb9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:32.768905 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c6eb7-be91-4efd-85b6-e835bb7aebb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:32.769189 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c6eb7-be91-4efd-85b6-e835bb7aebb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:32.836356 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c6eb7-be91-4efd-85b6-e835bb7aebb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:32.837801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c6eb7-be91-4efd-85b6-e835bb7aebb9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:32.855093 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:32.934518 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 549/1104] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:31:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:31:32.940435 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7cb4b-f2dd-429d-8d79-2247eb07ca39 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:33.001563 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:33.038891 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7cb4b-f2dd-429d-8d79-2247eb07ca39 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:33.072390 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fea26d-5ecb-4d89-bb1d-d826c058207d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:33.073017 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fea26d-5ecb-4d89-bb1d-d826c058207d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:33.073296 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fea26d-5ecb-4d89-bb1d-d826c058207d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:33.073564 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fea26d-5ecb-4d89-bb1d-d826c058207d None admin] PATH_INFO: `/v3/projects/bba64eea79c64df19fe3e59b9d745d1c` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:33.074686 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fea26d-5ecb-4d89-bb1d-d826c058207d None admin] RBAC: Authorizing `identity:get_project(project_id=bba64eea79c64df19fe3e59b9d745d1c)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:33.076184 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fea26d-5ecb-4d89-bb1d-d826c058207d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:33.077811 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 556/1105] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:31:32 2022] GET /identity/v3/projects/bba64eea79c64df19fe3e59b9d745d1c => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:33.084000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c6eb7-be91-4efd-85b6-e835bb7aebb9 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:33.182238 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c6eb7-be91-4efd-85b6-e835bb7aebb9 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:33.199955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:33.216339 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cfaef0-5b98-4b78-9e14-bec234d6e354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:33.216955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfaef0-5b98-4b78-9e14-bec234d6e354 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:33.217236 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfaef0-5b98-4b78-9e14-bec234d6e354 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:33.217510 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfaef0-5b98-4b78-9e14-bec234d6e354 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:33.218322 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cfaef0-5b98-4b78-9e14-bec234d6e354 None admin] RBAC: Adding query filter params (role.id=51b2d10dfffb4b75b186f3d8b15e2a78, scope.project.id=bba64eea79c64df19fe3e59b9d745d1c, user.id=ef69e9851acd4c60be511b1593c5a99a) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:33.218632 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cfaef0-5b98-4b78-9e14-bec234d6e354 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:33.220091 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cfaef0-5b98-4b78-9e14-bec234d6e354 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:33.225693 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 550/1106] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:31:33 2022] GET /identity/v3/role_assignments?user.id=ef69e9851acd4c60be511b1593c5a99a&scope.project.id=bba64eea79c64df19fe3e59b9d745d1c&role.id=51b2d10dfffb4b75b186f3d8b15e2a78 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:35.251020 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86498a63-ab29-483e-9549-b56731d8ad8f None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:35.251570 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86498a63-ab29-483e-9549-b56731d8ad8f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:35.251570 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86498a63-ab29-483e-9549-b56731d8ad8f None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:35.253493 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 557/1107] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:35.258563 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f4033-c0af-4af6-af31-2dadc50b3e1e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:35.258940 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f4033-c0af-4af6-af31-2dadc50b3e1e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:35.259121 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f4033-c0af-4af6-af31-2dadc50b3e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:35.277915 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-959f4033-c0af-4af6-af31-2dadc50b3e1e None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:35.279109 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-959f4033-c0af-4af6-af31-2dadc50b3e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:35.344006 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 551/1108] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:35.368414 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86498a63-ab29-483e-9549-b56731d8ad8f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:35.466452 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86498a63-ab29-483e-9549-b56731d8ad8f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:35.494757 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:35.499424 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269da5b-751e-4102-b253-96aeaff4cef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:35.500066 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269da5b-751e-4102-b253-96aeaff4cef5 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:35.500348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269da5b-751e-4102-b253-96aeaff4cef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:35.500620 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269da5b-751e-4102-b253-96aeaff4cef5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:35.501755 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0269da5b-751e-4102-b253-96aeaff4cef5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:35.503268 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0269da5b-751e-4102-b253-96aeaff4cef5 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:35.504858 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 558/1109] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:31:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:35.510523 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f4033-c0af-4af6-af31-2dadc50b3e1e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:35.606816 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f4033-c0af-4af6-af31-2dadc50b3e1e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:35.619915 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:35.641409 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10658fd7-042c-4150-bdb3-676b1c404cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:35.642027 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10658fd7-042c-4150-bdb3-676b1c404cdc None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:35.642307 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10658fd7-042c-4150-bdb3-676b1c404cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:35.642582 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10658fd7-042c-4150-bdb3-676b1c404cdc None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:35.643834 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10658fd7-042c-4150-bdb3-676b1c404cdc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:35.645266 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10658fd7-042c-4150-bdb3-676b1c404cdc None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:35.767865 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 552/1110] 10.222.0.97 () {60 vars in 1132 bytes} [Tue Jul 26 14:31:35 2022] POST /identity/v3/users => generated 286 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:31:37.781636 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852f7b2-7a4a-418b-8229-e359d77dea5f None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:37.782160 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852f7b2-7a4a-418b-8229-e359d77dea5f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:37.782160 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852f7b2-7a4a-418b-8229-e359d77dea5f None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:37.783621 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 559/1111] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:37.788835 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9fb9f-ed54-4d71-9ef6-12ff0c461681 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:37.789239 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9fb9f-ed54-4d71-9ef6-12ff0c461681 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:37.789395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9fb9f-ed54-4d71-9ef6-12ff0c461681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:37.807798 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-68c9fb9f-ed54-4d71-9ef6-12ff0c461681 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:37.808982 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-68c9fb9f-ed54-4d71-9ef6-12ff0c461681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:37.873258 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 553/1112] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:37.897022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852f7b2-7a4a-418b-8229-e359d77dea5f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:37.996541 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852f7b2-7a4a-418b-8229-e359d77dea5f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:38.029079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a3ea34-50a7-4687-afa2-fea4e9600433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:38.029704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3ea34-50a7-4687-afa2-fea4e9600433 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:38.029986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3ea34-50a7-4687-afa2-fea4e9600433 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:38.030262 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3ea34-50a7-4687-afa2-fea4e9600433 None admin] PATH_INFO: `/v3/roles/0602cf371b0f41b390e8a4499ae6bb3c` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:38.031765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a3ea34-50a7-4687-afa2-fea4e9600433 None admin] RBAC: Authorizing `identity:get_role(role_id=0602cf371b0f41b390e8a4499ae6bb3c)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:38.033065 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a3ea34-50a7-4687-afa2-fea4e9600433 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:38.034912 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 560/1113] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:31:37 2022] GET /identity/v3/roles/0602cf371b0f41b390e8a4499ae6bb3c => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:38.040722 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9fb9f-ed54-4d71-9ef6-12ff0c461681 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:38.132744 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:38.136136 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9fb9f-ed54-4d71-9ef6-12ff0c461681 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:38.168454 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05fac3-08ec-48de-a463-4ef8e6a5012b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:38.169088 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05fac3-08ec-48de-a463-4ef8e6a5012b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:38.169369 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05fac3-08ec-48de-a463-4ef8e6a5012b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:38.169642 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05fac3-08ec-48de-a463-4ef8e6a5012b None admin] PATH_INFO: `/v3/users/24f5f7a31899447e8aded9c496e0c0c8` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:38.182775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05fac3-08ec-48de-a463-4ef8e6a5012b None admin] RBAC: Authorizing `identity:get_user(user_id=24f5f7a31899447e8aded9c496e0c0c8)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:38.184355 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05fac3-08ec-48de-a463-4ef8e6a5012b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:38.186150 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 554/1114] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:31:38 2022] GET /identity/v3/users/24f5f7a31899447e8aded9c496e0c0c8 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:38.191963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a3ea34-50a7-4687-afa2-fea4e9600433 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:38.256439 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:38.288331 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a3ea34-50a7-4687-afa2-fea4e9600433 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:38.322748 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec19d1-3568-4b1e-82c2-10e577f477e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:38.323367 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec19d1-3568-4b1e-82c2-10e577f477e4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:38.323678 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec19d1-3568-4b1e-82c2-10e577f477e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:38.323953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec19d1-3568-4b1e-82c2-10e577f477e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:38.388825 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec19d1-3568-4b1e-82c2-10e577f477e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:38.390259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec19d1-3568-4b1e-82c2-10e577f477e4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:38.451577 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:38.484407 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 561/1115] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:31:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:31:38.490892 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05fac3-08ec-48de-a463-4ef8e6a5012b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:38.587533 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05fac3-08ec-48de-a463-4ef8e6a5012b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:38.619819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7280e36-2196-4fc0-8b61-8edb7afe8904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:38.620445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7280e36-2196-4fc0-8b61-8edb7afe8904 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:38.620721 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7280e36-2196-4fc0-8b61-8edb7afe8904 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:38.620992 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7280e36-2196-4fc0-8b61-8edb7afe8904 None admin] PATH_INFO: `/v3/projects/bba64eea79c64df19fe3e59b9d745d1c` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:38.622137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7280e36-2196-4fc0-8b61-8edb7afe8904 None admin] RBAC: Authorizing `identity:get_project(project_id=bba64eea79c64df19fe3e59b9d745d1c)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:38.623598 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7280e36-2196-4fc0-8b61-8edb7afe8904 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:38.625231 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 555/1116] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:31:38 2022] GET /identity/v3/projects/bba64eea79c64df19fe3e59b9d745d1c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:38.631886 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec19d1-3568-4b1e-82c2-10e577f477e4 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:38.728679 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec19d1-3568-4b1e-82c2-10e577f477e4 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:38.761965 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e056ca6-89c9-4270-9679-b4932f4d5814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:38.762838 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e056ca6-89c9-4270-9679-b4932f4d5814 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:38.763122 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e056ca6-89c9-4270-9679-b4932f4d5814 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:38.763423 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e056ca6-89c9-4270-9679-b4932f4d5814 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:38.764202 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e056ca6-89c9-4270-9679-b4932f4d5814 None admin] RBAC: Adding query filter params (role.id=0602cf371b0f41b390e8a4499ae6bb3c, scope.project.id=bba64eea79c64df19fe3e59b9d745d1c, user.id=24f5f7a31899447e8aded9c496e0c0c8) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:38.764512 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e056ca6-89c9-4270-9679-b4932f4d5814 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:38.765924 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e056ca6-89c9-4270-9679-b4932f4d5814 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:38.771798 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 562/1117] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:31:38 2022] GET /identity/v3/role_assignments?user.id=24f5f7a31899447e8aded9c496e0c0c8&scope.project.id=bba64eea79c64df19fe3e59b9d745d1c&role.id=0602cf371b0f41b390e8a4499ae6bb3c => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:40.718907 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899cf5bd-fd21-45e9-95b1-ee33a4296824 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:40.719479 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899cf5bd-fd21-45e9-95b1-ee33a4296824 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:40.719479 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899cf5bd-fd21-45e9-95b1-ee33a4296824 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:40.720857 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 556/1118] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:40.726039 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271262a-497a-48de-b6b2-cffc02b5a4ee None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:40.726445 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271262a-497a-48de-b6b2-cffc02b5a4ee None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:40.726586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271262a-497a-48de-b6b2-cffc02b5a4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:40.744924 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-4271262a-497a-48de-b6b2-cffc02b5a4ee None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:40.746119 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4271262a-497a-48de-b6b2-cffc02b5a4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:40.771582 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:40.817912 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 563/1119] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 14:31:40.840571 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899cf5bd-fd21-45e9-95b1-ee33a4296824 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:40.865525 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:40.940847 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899cf5bd-fd21-45e9-95b1-ee33a4296824 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:40.973586 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a6bf3c-b9ba-4d91-b7b1-ecc6e9100bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:40.974150 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6bf3c-b9ba-4d91-b7b1-ecc6e9100bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:40.974426 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6bf3c-b9ba-4d91-b7b1-ecc6e9100bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:40.974697 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6bf3c-b9ba-4d91-b7b1-ecc6e9100bf2 None admin] PATH_INFO: `/v3/roles/0602cf371b0f41b390e8a4499ae6bb3c` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:40.976135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a6bf3c-b9ba-4d91-b7b1-ecc6e9100bf2 None admin] RBAC: Authorizing `identity:get_role(role_id=0602cf371b0f41b390e8a4499ae6bb3c)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:40.977506 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a6bf3c-b9ba-4d91-b7b1-ecc6e9100bf2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:40.978828 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 557/1120] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:31:40 2022] GET /identity/v3/roles/0602cf371b0f41b390e8a4499ae6bb3c => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:40.984748 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271262a-497a-48de-b6b2-cffc02b5a4ee None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:41.081165 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271262a-497a-48de-b6b2-cffc02b5a4ee None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:41.114340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b355ed-140a-46f5-820a-69ace04c9240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:41.114969 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b355ed-140a-46f5-820a-69ace04c9240 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:41.115250 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b355ed-140a-46f5-820a-69ace04c9240 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:41.115558 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b355ed-140a-46f5-820a-69ace04c9240 None admin] PATH_INFO: `/v3/users/24f5f7a31899447e8aded9c496e0c0c8` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:41.116703 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b355ed-140a-46f5-820a-69ace04c9240 None admin] RBAC: Authorizing `identity:get_user(user_id=24f5f7a31899447e8aded9c496e0c0c8)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:41.118126 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b355ed-140a-46f5-820a-69ace04c9240 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:41.119786 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 564/1121] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:31:40 2022] GET /identity/v3/users/24f5f7a31899447e8aded9c496e0c0c8 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:41.125621 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a6bf3c-b9ba-4d91-b7b1-ecc6e9100bf2 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:41.217696 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:41.232844 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a6bf3c-b9ba-4d91-b7b1-ecc6e9100bf2 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:41.262501 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dfed8f-a489-450d-947b-1cb73c5f1865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:41.263144 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dfed8f-a489-450d-947b-1cb73c5f1865 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:41.263451 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dfed8f-a489-450d-947b-1cb73c5f1865 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:41.263729 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dfed8f-a489-450d-947b-1cb73c5f1865 None admin] PATH_INFO: `/v3/projects/bba64eea79c64df19fe3e59b9d745d1c` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:41.264885 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dfed8f-a489-450d-947b-1cb73c5f1865 None admin] RBAC: Authorizing `identity:get_project(project_id=bba64eea79c64df19fe3e59b9d745d1c)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:41.266314 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dfed8f-a489-450d-947b-1cb73c5f1865 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:41.268658 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 558/1122] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:31:41 2022] GET /identity/v3/projects/bba64eea79c64df19fe3e59b9d745d1c => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:41.274850 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b355ed-140a-46f5-820a-69ace04c9240 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:41.304063 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:41.372352 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b355ed-140a-46f5-820a-69ace04c9240 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:41.405725 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac077b6c-6153-49bf-95d1-a19c5b4d11f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:41.406386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac077b6c-6153-49bf-95d1-a19c5b4d11f0 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:41.406671 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac077b6c-6153-49bf-95d1-a19c5b4d11f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:41.406948 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac077b6c-6153-49bf-95d1-a19c5b4d11f0 None admin] PATH_INFO: `/v3/projects/bba64eea79c64df19fe3e59b9d745d1c/users/24f5f7a31899447e8aded9c496e0c0c8/roles/0602cf371b0f41b390e8a4499ae6bb3c` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:41.409197 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac077b6c-6153-49bf-95d1-a19c5b4d11f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=bba64eea79c64df19fe3e59b9d745d1c, user_id=24f5f7a31899447e8aded9c496e0c0c8, role_id=0602cf371b0f41b390e8a4499ae6bb3c)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:41.410627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac077b6c-6153-49bf-95d1-a19c5b4d11f0 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:41.504432 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 565/1123] 10.222.0.97 () {58 vars in 1439 bytes} [Tue Jul 26 14:31:41 2022] PUT /identity/v3/projects/bba64eea79c64df19fe3e59b9d745d1c/users/24f5f7a31899447e8aded9c496e0c0c8/roles/0602cf371b0f41b390e8a4499ae6bb3c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:31:43.468590 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a789fcc-0ef6-4f3e-a5de-813d1d7e5df0 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:43.469116 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a789fcc-0ef6-4f3e-a5de-813d1d7e5df0 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:43.469116 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a789fcc-0ef6-4f3e-a5de-813d1d7e5df0 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:43.470427 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 559/1124] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:43.475739 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d37ada-dfba-4b7c-bafd-e689f33cbcf8 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:43.476145 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d37ada-dfba-4b7c-bafd-e689f33cbcf8 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:43.476294 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d37ada-dfba-4b7c-bafd-e689f33cbcf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:43.493938 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-32d37ada-dfba-4b7c-bafd-e689f33cbcf8 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:43.495111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-32d37ada-dfba-4b7c-bafd-e689f33cbcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:43.559637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 566/1125] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:43.583342 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a789fcc-0ef6-4f3e-a5de-813d1d7e5df0 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:43.696984 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a789fcc-0ef6-4f3e-a5de-813d1d7e5df0 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:43.730493 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f9afa-0cb5-4b72-841e-73a89869ed74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:43.731064 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f9afa-0cb5-4b72-841e-73a89869ed74 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:43.731347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f9afa-0cb5-4b72-841e-73a89869ed74 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:43.731653 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f9afa-0cb5-4b72-841e-73a89869ed74 None admin] PATH_INFO: `/v3/roles/0602cf371b0f41b390e8a4499ae6bb3c` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:43.733078 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f9afa-0cb5-4b72-841e-73a89869ed74 None admin] RBAC: Authorizing `identity:get_role(role_id=0602cf371b0f41b390e8a4499ae6bb3c)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:43.734430 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f9afa-0cb5-4b72-841e-73a89869ed74 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:43.735815 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 560/1126] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:31:43 2022] GET /identity/v3/roles/0602cf371b0f41b390e8a4499ae6bb3c => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:43.741655 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d37ada-dfba-4b7c-bafd-e689f33cbcf8 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:43.762261 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:43.839767 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d37ada-dfba-4b7c-bafd-e689f33cbcf8 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:43.873228 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fc4c7-701d-4b82-9ede-eaf1878428b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:43.873888 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fc4c7-701d-4b82-9ede-eaf1878428b5 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:43.874191 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fc4c7-701d-4b82-9ede-eaf1878428b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:43.874494 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fc4c7-701d-4b82-9ede-eaf1878428b5 None admin] PATH_INFO: `/v3/users/24f5f7a31899447e8aded9c496e0c0c8` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:43.875692 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5fc4c7-701d-4b82-9ede-eaf1878428b5 None admin] RBAC: Authorizing `identity:get_user(user_id=24f5f7a31899447e8aded9c496e0c0c8)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:43.877238 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5fc4c7-701d-4b82-9ede-eaf1878428b5 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:43.878937 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 567/1127] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:31:43 2022] GET /identity/v3/users/24f5f7a31899447e8aded9c496e0c0c8 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:43.884710 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f9afa-0cb5-4b72-841e-73a89869ed74 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:43.913696 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:43.983154 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f9afa-0cb5-4b72-841e-73a89869ed74 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:44.016958 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29678ce8-f521-4bc8-946d-4c22ee36f3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:44.017605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29678ce8-f521-4bc8-946d-4c22ee36f3a2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:44.017886 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29678ce8-f521-4bc8-946d-4c22ee36f3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:44.018164 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29678ce8-f521-4bc8-946d-4c22ee36f3a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:44.083445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29678ce8-f521-4bc8-946d-4c22ee36f3a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:44.084875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29678ce8-f521-4bc8-946d-4c22ee36f3a2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:44.110439 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:44.179820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 561/1128] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:31:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 14:31:44.185712 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fc4c7-701d-4b82-9ede-eaf1878428b5 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:44.270130 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:44.281313 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fc4c7-701d-4b82-9ede-eaf1878428b5 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:44.313657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe772292-a636-4a44-b788-c63d36a7d824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:44.314809 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe772292-a636-4a44-b788-c63d36a7d824 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:44.315093 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe772292-a636-4a44-b788-c63d36a7d824 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:44.315365 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe772292-a636-4a44-b788-c63d36a7d824 None admin] PATH_INFO: `/v3/projects/bba64eea79c64df19fe3e59b9d745d1c` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:44.316533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe772292-a636-4a44-b788-c63d36a7d824 None admin] RBAC: Authorizing `identity:get_project(project_id=bba64eea79c64df19fe3e59b9d745d1c)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:44.317943 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe772292-a636-4a44-b788-c63d36a7d824 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:44.319596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 568/1129] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:31:44 2022] GET /identity/v3/projects/bba64eea79c64df19fe3e59b9d745d1c => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:44.325556 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29678ce8-f521-4bc8-946d-4c22ee36f3a2 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:44.421699 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29678ce8-f521-4bc8-946d-4c22ee36f3a2 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:44.447798 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:44.455388 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1793b0-ed7d-4471-ac8e-87f0d68d9341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:44.456034 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1793b0-ed7d-4471-ac8e-87f0d68d9341 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:44.456318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1793b0-ed7d-4471-ac8e-87f0d68d9341 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:44.456593 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1793b0-ed7d-4471-ac8e-87f0d68d9341 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:44.457373 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1793b0-ed7d-4471-ac8e-87f0d68d9341 None admin] RBAC: Adding query filter params (role.id=0602cf371b0f41b390e8a4499ae6bb3c, scope.project.id=bba64eea79c64df19fe3e59b9d745d1c, user.id=24f5f7a31899447e8aded9c496e0c0c8) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:44.457685 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1793b0-ed7d-4471-ac8e-87f0d68d9341 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:44.459103 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1793b0-ed7d-4471-ac8e-87f0d68d9341 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:44.464701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 562/1130] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:31:44 2022] GET /identity/v3/role_assignments?user.id=24f5f7a31899447e8aded9c496e0c0c8&scope.project.id=bba64eea79c64df19fe3e59b9d745d1c&role.id=0602cf371b0f41b390e8a4499ae6bb3c => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:46.459697 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b63bb8-425a-4b30-a284-6d1078a4ebd1 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:46.460195 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b63bb8-425a-4b30-a284-6d1078a4ebd1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:46.460195 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b63bb8-425a-4b30-a284-6d1078a4ebd1 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:46.461512 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 569/1131] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:46.466743 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39526b-6198-4724-ada6-be03992cbf1a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:46.467117 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39526b-6198-4724-ada6-be03992cbf1a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:46.467304 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39526b-6198-4724-ada6-be03992cbf1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:46.485409 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-be39526b-6198-4724-ada6-be03992cbf1a None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:46.487273 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-be39526b-6198-4724-ada6-be03992cbf1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:46.552939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 563/1132] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:46.575748 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b63bb8-425a-4b30-a284-6d1078a4ebd1 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:46.676021 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b63bb8-425a-4b30-a284-6d1078a4ebd1 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:46.709345 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f8b89-3a47-4c99-baba-68c46f5245eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:46.710024 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f8b89-3a47-4c99-baba-68c46f5245eb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:46.710253 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f8b89-3a47-4c99-baba-68c46f5245eb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:46.710529 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f8b89-3a47-4c99-baba-68c46f5245eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:46.711689 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f8b89-3a47-4c99-baba-68c46f5245eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:46.713113 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f8b89-3a47-4c99-baba-68c46f5245eb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:46.714655 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 570/1133] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:31:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:46.720531 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39526b-6198-4724-ada6-be03992cbf1a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:46.816200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39526b-6198-4724-ada6-be03992cbf1a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:46.838214 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:46.849255 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f4fb5-aedf-4ea2-8c39-b20c453f3b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:46.849870 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f4fb5-aedf-4ea2-8c39-b20c453f3b08 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:46.850154 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f4fb5-aedf-4ea2-8c39-b20c453f3b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:46.851149 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f4fb5-aedf-4ea2-8c39-b20c453f3b08 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:46.851149 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f4fb5-aedf-4ea2-8c39-b20c453f3b08 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:46.852728 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f4fb5-aedf-4ea2-8c39-b20c453f3b08 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:46.950490 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 564/1134] 10.222.0.97 () {60 vars in 1141 bytes} [Tue Jul 26 14:31:46 2022] POST /identity/v3/projects => generated 311 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:31:48.986322 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ae503-0fb0-410d-b00b-193c4e775f8b None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:48.986956 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ae503-0fb0-410d-b00b-193c4e775f8b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:48.986956 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ae503-0fb0-410d-b00b-193c4e775f8b None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:48.988263 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 571/1135] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:48.993672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036d73e-c1dc-4364-9a2f-81b95a542995 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:48.994134 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036d73e-c1dc-4364-9a2f-81b95a542995 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:48.994226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036d73e-c1dc-4364-9a2f-81b95a542995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:49.012386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3036d73e-c1dc-4364-9a2f-81b95a542995 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:49.013572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3036d73e-c1dc-4364-9a2f-81b95a542995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:49.080719 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 565/1136] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:49.105695 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ae503-0fb0-410d-b00b-193c4e775f8b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:49.130468 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:49.206330 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ae503-0fb0-410d-b00b-193c4e775f8b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:49.239978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520bb0cf-4ede-47a2-9a75-b3b95329b030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:49.240542 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520bb0cf-4ede-47a2-9a75-b3b95329b030 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:49.240827 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520bb0cf-4ede-47a2-9a75-b3b95329b030 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:49.241103 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520bb0cf-4ede-47a2-9a75-b3b95329b030 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:49.242255 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520bb0cf-4ede-47a2-9a75-b3b95329b030 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:49.243704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520bb0cf-4ede-47a2-9a75-b3b95329b030 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:49.245288 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 572/1137] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:31:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:49.251147 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036d73e-c1dc-4364-9a2f-81b95a542995 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:49.348571 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036d73e-c1dc-4364-9a2f-81b95a542995 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:49.350871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:49.381668 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da99112-a683-4777-9b6e-08148cea2fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:49.382325 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da99112-a683-4777-9b6e-08148cea2fbe None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:49.382608 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da99112-a683-4777-9b6e-08148cea2fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:49.382883 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da99112-a683-4777-9b6e-08148cea2fbe None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:49.383588 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da99112-a683-4777-9b6e-08148cea2fbe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:49.385064 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da99112-a683-4777-9b6e-08148cea2fbe None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:49.526629 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 566/1138] 10.222.0.97 () {60 vars in 1132 bytes} [Tue Jul 26 14:31:49 2022] POST /identity/v3/users => generated 286 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:31:51.543441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185ff56-4f6b-42c8-ba0b-0062ef1e9188 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:51.544003 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185ff56-4f6b-42c8-ba0b-0062ef1e9188 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:51.544003 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185ff56-4f6b-42c8-ba0b-0062ef1e9188 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:51.545318 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 573/1139] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:51.550617 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66807e5a-af82-4325-b142-06150508d3b1 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:51.550995 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66807e5a-af82-4325-b142-06150508d3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:51.551171 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66807e5a-af82-4325-b142-06150508d3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:51.570102 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-66807e5a-af82-4325-b142-06150508d3b1 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:51.571272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-66807e5a-af82-4325-b142-06150508d3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:51.637092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 567/1140] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:51.660428 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0185ff56-4f6b-42c8-ba0b-0062ef1e9188 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:51.750116 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:51.759421 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0185ff56-4f6b-42c8-ba0b-0062ef1e9188 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:51.793676 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53ed41-59b7-41e7-b651-f7c7393651a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:51.794246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53ed41-59b7-41e7-b651-f7c7393651a6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:51.794529 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53ed41-59b7-41e7-b651-f7c7393651a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:51.794802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53ed41-59b7-41e7-b651-f7c7393651a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:51.805312 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53ed41-59b7-41e7-b651-f7c7393651a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:31:51.810152 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-bf53ed41-59b7-41e7-b651-f7c7393651a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:31:51.810152 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:51.810152 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:51.810152 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:51.810152 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:51.810152 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:51.810152 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:51.810152 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:51.810152 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:51.810152 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:51.810152 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:51.810152 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:51.810152 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:31:51.810152 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:31:51.810152 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:31:51.810152 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:31:51.811348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:31:51.811348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:31:51.811348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:31:51.811348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:31:51.811348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:51.811348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:51.811348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:51.811348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:51.811348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:51.811348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:51.811348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:51.811348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:51.811348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:51.811348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:51.811348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:51.811348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:51.812531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:51.812531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:51.812531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:51.812531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:51.812531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:31:51.812531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:31:51.812531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:31:51.812531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:31:51.812531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:31:51.812531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:31:51.812531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:31:51.812531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:51.812531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 574/1141] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:31:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:31:51.818158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66807e5a-af82-4325-b142-06150508d3b1 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:51.904932 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:51.914118 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66807e5a-af82-4325-b142-06150508d3b1 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:51.948887 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb2566-b690-4d30-ba1b-82fca8da7713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:51.949525 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb2566-b690-4d30-ba1b-82fca8da7713 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:51.949808 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb2566-b690-4d30-ba1b-82fca8da7713 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:51.950084 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb2566-b690-4d30-ba1b-82fca8da7713 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:51.950813 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb2566-b690-4d30-ba1b-82fca8da7713 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:51.951157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb2566-b690-4d30-ba1b-82fca8da7713 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:51.952584 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb2566-b690-4d30-ba1b-82fca8da7713 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:51.962299 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 568/1142] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:31:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:51.968354 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53ed41-59b7-41e7-b651-f7c7393651a6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:52.067716 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53ed41-59b7-41e7-b651-f7c7393651a6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:52.101376 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71135982-87c5-4c83-98f8-5e2d11d321d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:52.102013 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71135982-87c5-4c83-98f8-5e2d11d321d4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:52.102297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71135982-87c5-4c83-98f8-5e2d11d321d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:52.102573 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71135982-87c5-4c83-98f8-5e2d11d321d4 None admin] PATH_INFO: `/v3/users/277e8f2ab107459a880f3963f626cc6e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:52.116713 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71135982-87c5-4c83-98f8-5e2d11d321d4 None admin] RBAC: Authorizing `identity:get_user(user_id=277e8f2ab107459a880f3963f626cc6e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:52.118168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71135982-87c5-4c83-98f8-5e2d11d321d4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:52.119990 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 575/1143] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:31:51 2022] GET /identity/v3/users/277e8f2ab107459a880f3963f626cc6e => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:52.125721 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb2566-b690-4d30-ba1b-82fca8da7713 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:52.230894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb2566-b690-4d30-ba1b-82fca8da7713 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:52.263972 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1df9d0-f302-4ee2-8b28-3e1168db44f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:52.264584 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1df9d0-f302-4ee2-8b28-3e1168db44f6 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:52.264867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1df9d0-f302-4ee2-8b28-3e1168db44f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:52.265139 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1df9d0-f302-4ee2-8b28-3e1168db44f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:52.285979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:52.340107 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1df9d0-f302-4ee2-8b28-3e1168db44f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:52.341477 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1df9d0-f302-4ee2-8b28-3e1168db44f6 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:52.442689 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 569/1144] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:31:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:31:52.448657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71135982-87c5-4c83-98f8-5e2d11d321d4 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:52.460805 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:52.546603 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71135982-87c5-4c83-98f8-5e2d11d321d4 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:52.579576 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248164b-997e-4400-ba55-e1ead1299ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:52.580205 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248164b-997e-4400-ba55-e1ead1299ffa None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:52.580491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248164b-997e-4400-ba55-e1ead1299ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:52.580763 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248164b-997e-4400-ba55-e1ead1299ffa None admin] PATH_INFO: `/v3/projects/0ecd3871ccd44830b009fd32136601ae` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:52.581930 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2248164b-997e-4400-ba55-e1ead1299ffa None admin] RBAC: Authorizing `identity:get_project(project_id=0ecd3871ccd44830b009fd32136601ae)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:52.583336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2248164b-997e-4400-ba55-e1ead1299ffa None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:52.585013 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 576/1145] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:31:52 2022] GET /identity/v3/projects/0ecd3871ccd44830b009fd32136601ae => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:52.591230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1df9d0-f302-4ee2-8b28-3e1168db44f6 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:52.643360 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:52.690445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1df9d0-f302-4ee2-8b28-3e1168db44f6 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:52.724129 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be639981-d8ba-4e28-a8d4-7154412b255e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:52.724751 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be639981-d8ba-4e28-a8d4-7154412b255e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:52.725038 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be639981-d8ba-4e28-a8d4-7154412b255e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:52.725315 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be639981-d8ba-4e28-a8d4-7154412b255e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:52.726099 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be639981-d8ba-4e28-a8d4-7154412b255e None admin] RBAC: Adding query filter params (role.id=51b2d10dfffb4b75b186f3d8b15e2a78, scope.project.id=0ecd3871ccd44830b009fd32136601ae, user.id=277e8f2ab107459a880f3963f626cc6e) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:52.726411 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be639981-d8ba-4e28-a8d4-7154412b255e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:52.727893 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be639981-d8ba-4e28-a8d4-7154412b255e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:52.733278 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 570/1146] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:31:52 2022] GET /identity/v3/role_assignments?user.id=277e8f2ab107459a880f3963f626cc6e&scope.project.id=0ecd3871ccd44830b009fd32136601ae&role.id=51b2d10dfffb4b75b186f3d8b15e2a78 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:54.736700 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea33024-e387-49d2-8377-6720d934637f None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:54.737256 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea33024-e387-49d2-8377-6720d934637f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:54.737256 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea33024-e387-49d2-8377-6720d934637f None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:54.739088 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 577/1147] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:54.744341 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dabfa-734c-4845-b911-47844e58bc37 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:54.744754 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dabfa-734c-4845-b911-47844e58bc37 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:54.744896 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dabfa-734c-4845-b911-47844e58bc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:54.762677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-2b0dabfa-734c-4845-b911-47844e58bc37 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:54.763878 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2b0dabfa-734c-4845-b911-47844e58bc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:54.834072 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 571/1148] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:54.855992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea33024-e387-49d2-8377-6720d934637f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:54.934737 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:54.956256 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea33024-e387-49d2-8377-6720d934637f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:54.989026 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67633e1d-fa66-4bba-8b08-9261dbba856e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:54.989600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67633e1d-fa66-4bba-8b08-9261dbba856e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:54.989890 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67633e1d-fa66-4bba-8b08-9261dbba856e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:54.990166 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67633e1d-fa66-4bba-8b08-9261dbba856e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:55.000083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67633e1d-fa66-4bba-8b08-9261dbba856e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:31:55.005505 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-67633e1d-fa66-4bba-8b08-9261dbba856e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:31:55.005505 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:55.005505 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:55.005505 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:55.005505 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:55.005505 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:55.005505 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:55.005505 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:55.005505 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:55.005505 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:55.005505 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:55.005505 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:55.005505 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:31:55.005505 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:31:55.005505 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:31:55.005505 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:31:55.006637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:31:55.006637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:31:55.006637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:31:55.006637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:31:55.006637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:55.006637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:55.006637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:55.006637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:55.006637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:55.006637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:55.006637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:55.006637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:55.006637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:55.006637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:55.006637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:55.006637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:55.007675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:55.007675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:55.007675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:55.007675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:55.007675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:31:55.007675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:31:55.007675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:31:55.007675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:31:55.007675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:31:55.007675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:31:55.007675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:31:55.007675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:31:55.007675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 578/1149] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:31:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:31:55.012939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0dabfa-734c-4845-b911-47844e58bc37 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:55.058993 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:55.111924 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0dabfa-734c-4845-b911-47844e58bc37 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:55.145109 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252d85b-9477-44ca-a044-87a5a7526090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:55.145726 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252d85b-9477-44ca-a044-87a5a7526090 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:55.146010 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252d85b-9477-44ca-a044-87a5a7526090 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:55.146293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252d85b-9477-44ca-a044-87a5a7526090 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:55.147000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252d85b-9477-44ca-a044-87a5a7526090 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:55.147305 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252d85b-9477-44ca-a044-87a5a7526090 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:55.148703 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252d85b-9477-44ca-a044-87a5a7526090 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:55.156722 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 572/1150] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:31:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:55.163196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67633e1d-fa66-4bba-8b08-9261dbba856e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:55.268590 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67633e1d-fa66-4bba-8b08-9261dbba856e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:55.301476 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253fa77-7380-4e43-8ac6-1389d42a818b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:55.302119 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253fa77-7380-4e43-8ac6-1389d42a818b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:55.302404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253fa77-7380-4e43-8ac6-1389d42a818b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:55.302684 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253fa77-7380-4e43-8ac6-1389d42a818b None admin] PATH_INFO: `/v3/users/277e8f2ab107459a880f3963f626cc6e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:55.303874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9253fa77-7380-4e43-8ac6-1389d42a818b None admin] RBAC: Authorizing `identity:get_user(user_id=277e8f2ab107459a880f3963f626cc6e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:55.305319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9253fa77-7380-4e43-8ac6-1389d42a818b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:55.306921 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 579/1151] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:31:55 2022] GET /identity/v3/users/277e8f2ab107459a880f3963f626cc6e => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:55.312902 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252d85b-9477-44ca-a044-87a5a7526090 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:55.414950 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252d85b-9477-44ca-a044-87a5a7526090 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:55.424092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:55.448651 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf2db28-6c92-4d11-a93d-357c321577b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:55.449295 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf2db28-6c92-4d11-a93d-357c321577b2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:55.449579 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf2db28-6c92-4d11-a93d-357c321577b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:55.449859 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf2db28-6c92-4d11-a93d-357c321577b2 None admin] PATH_INFO: `/v3/projects/0ecd3871ccd44830b009fd32136601ae` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:55.451009 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf2db28-6c92-4d11-a93d-357c321577b2 None admin] RBAC: Authorizing `identity:get_project(project_id=0ecd3871ccd44830b009fd32136601ae)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:55.452495 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf2db28-6c92-4d11-a93d-357c321577b2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:55.454764 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 573/1152] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:31:55 2022] GET /identity/v3/projects/0ecd3871ccd44830b009fd32136601ae => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:55.460824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253fa77-7380-4e43-8ac6-1389d42a818b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:55.470383 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:55.561729 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253fa77-7380-4e43-8ac6-1389d42a818b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:55.595645 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae90796-0a18-4548-b3f5-437a23b40f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:55.596344 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae90796-0a18-4548-b3f5-437a23b40f10 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:55.596629 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae90796-0a18-4548-b3f5-437a23b40f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:55.596907 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae90796-0a18-4548-b3f5-437a23b40f10 None admin] PATH_INFO: `/v3/projects/0ecd3871ccd44830b009fd32136601ae/users/277e8f2ab107459a880f3963f626cc6e/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:55.598826 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae90796-0a18-4548-b3f5-437a23b40f10 None admin] RBAC: Authorizing `identity:create_grant(project_id=0ecd3871ccd44830b009fd32136601ae, user_id=277e8f2ab107459a880f3963f626cc6e, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:55.600280 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae90796-0a18-4548-b3f5-437a23b40f10 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:55.714241 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 580/1153] 10.222.0.97 () {58 vars in 1439 bytes} [Tue Jul 26 14:31:55 2022] PUT /identity/v3/projects/0ecd3871ccd44830b009fd32136601ae/users/277e8f2ab107459a880f3963f626cc6e/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:31:57.699211 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0c408-54dc-40d8-8e12-b234b8bf3ce1 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:57.699803 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0c408-54dc-40d8-8e12-b234b8bf3ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:57.699803 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0c408-54dc-40d8-8e12-b234b8bf3ce1 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:57.701154 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 574/1154] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:31:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:31:57.706428 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdd9d9-b4c5-4e5c-a016-5f7949bbeb07 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:57.706824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdd9d9-b4c5-4e5c-a016-5f7949bbeb07 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:57.706986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdd9d9-b4c5-4e5c-a016-5f7949bbeb07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:57.725262 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-4fbdd9d9-b4c5-4e5c-a016-5f7949bbeb07 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:31:57.726446 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4fbdd9d9-b4c5-4e5c-a016-5f7949bbeb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:57.784695 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:57.791954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 581/1155] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:31:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:31:57.816086 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda0c408-54dc-40d8-8e12-b234b8bf3ce1 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:57.914349 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda0c408-54dc-40d8-8e12-b234b8bf3ce1 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:57.947441 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1f019-c8ea-4e47-a53a-a96e8c1d7fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:57.948081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1f019-c8ea-4e47-a53a-a96e8c1d7fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:57.948366 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1f019-c8ea-4e47-a53a-a96e8c1d7fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:57.948641 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1f019-c8ea-4e47-a53a-a96e8c1d7fb6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:57.959198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf1f019-c8ea-4e47-a53a-a96e8c1d7fb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:31:57.964029 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-bdf1f019-c8ea-4e47-a53a-a96e8c1d7fb6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:31:57.964029 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:31:57.964029 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:31:57.964029 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:31:57.964029 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:31:57.964029 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:31:57.964029 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:31:57.964029 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:31:57.964029 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:31:57.964029 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:31:57.964029 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:31:57.964029 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:31:57.964029 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:31:57.964029 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:31:57.964029 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:31:57.964029 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:31:57.965085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:31:57.965085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:31:57.965085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:31:57.965085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:31:57.965085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:31:57.965085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:31:57.965085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:31:57.965085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:31:57.965085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:31:57.965085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:31:57.965085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:31:57.965085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:31:57.965085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:31:57.965085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:31:57.965085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:31:57.965085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:31:57.965951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:31:57.965951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:31:57.965951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:31:57.965951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:31:57.965951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:31:57.965951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:31:57.965951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:31:57.965951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:31:57.965951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:31:57.965951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:31:57.965951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:31:57.965951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:31:57.965951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 575/1156] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:31:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:31:57.971214 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbdd9d9-b4c5-4e5c-a016-5f7949bbeb07 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:58.071640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbdd9d9-b4c5-4e5c-a016-5f7949bbeb07 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:58.103755 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3756b8d-bfbe-424a-836a-edbecded9a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:58.104357 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3756b8d-bfbe-424a-836a-edbecded9a5b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:58.104636 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3756b8d-bfbe-424a-836a-edbecded9a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:58.104910 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3756b8d-bfbe-424a-836a-edbecded9a5b None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:58.105611 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3756b8d-bfbe-424a-836a-edbecded9a5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:58.105915 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3756b8d-bfbe-424a-836a-edbecded9a5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:58.107282 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3756b8d-bfbe-424a-836a-edbecded9a5b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:58.115860 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 582/1157] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:31:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:58.121316 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1f019-c8ea-4e47-a53a-a96e8c1d7fb6 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:58.155826 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:58.220127 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1f019-c8ea-4e47-a53a-a96e8c1d7fb6 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:58.253517 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2eec62-c039-43ac-8447-9c005a4246d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:58.254146 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2eec62-c039-43ac-8447-9c005a4246d5 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:58.254429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2eec62-c039-43ac-8447-9c005a4246d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:58.254706 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2eec62-c039-43ac-8447-9c005a4246d5 None admin] PATH_INFO: `/v3/users/277e8f2ab107459a880f3963f626cc6e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:58.255884 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2eec62-c039-43ac-8447-9c005a4246d5 None admin] RBAC: Authorizing `identity:get_user(user_id=277e8f2ab107459a880f3963f626cc6e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:58.257305 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2eec62-c039-43ac-8447-9c005a4246d5 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:58.259012 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 576/1158] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:31:58 2022] GET /identity/v3/users/277e8f2ab107459a880f3963f626cc6e => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:58.265191 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3756b8d-bfbe-424a-836a-edbecded9a5b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:58.321726 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:58.362640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3756b8d-bfbe-424a-836a-edbecded9a5b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:58.394184 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a97f9-cc8c-4b57-8495-755241967614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:58.394784 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a97f9-cc8c-4b57-8495-755241967614 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:58.395063 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a97f9-cc8c-4b57-8495-755241967614 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:58.395335 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a97f9-cc8c-4b57-8495-755241967614 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:58.459948 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87a97f9-cc8c-4b57-8495-755241967614 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:58.461372 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87a97f9-cc8c-4b57-8495-755241967614 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:58.514297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:58.556052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 583/1159] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:31:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:31:58.562042 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2eec62-c039-43ac-8447-9c005a4246d5 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:58.662171 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2eec62-c039-43ac-8447-9c005a4246d5 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:58.664495 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:58.695504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2661f81e-8ab4-434b-817f-4d015b129ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:58.696137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661f81e-8ab4-434b-817f-4d015b129ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:58.696425 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661f81e-8ab4-434b-817f-4d015b129ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:58.696704 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661f81e-8ab4-434b-817f-4d015b129ae3 None admin] PATH_INFO: `/v3/projects/0ecd3871ccd44830b009fd32136601ae` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:58.697918 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2661f81e-8ab4-434b-817f-4d015b129ae3 None admin] RBAC: Authorizing `identity:get_project(project_id=0ecd3871ccd44830b009fd32136601ae)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:58.699376 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2661f81e-8ab4-434b-817f-4d015b129ae3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:58.701202 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 577/1160] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:31:58 2022] GET /identity/v3/projects/0ecd3871ccd44830b009fd32136601ae => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:31:58.707900 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a97f9-cc8c-4b57-8495-755241967614 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:58.802953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a97f9-cc8c-4b57-8495-755241967614 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:58.835444 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d873bf2-a5a2-48d5-9aaa-a0de4a5f3a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:58.836048 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d873bf2-a5a2-48d5-9aaa-a0de4a5f3a25 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:58.836329 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d873bf2-a5a2-48d5-9aaa-a0de4a5f3a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:58.836600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d873bf2-a5a2-48d5-9aaa-a0de4a5f3a25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:58.837374 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d873bf2-a5a2-48d5-9aaa-a0de4a5f3a25 None admin] RBAC: Adding query filter params (role.id=51b2d10dfffb4b75b186f3d8b15e2a78, scope.project.id=0ecd3871ccd44830b009fd32136601ae, user.id=277e8f2ab107459a880f3963f626cc6e) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:31:58.837683 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d873bf2-a5a2-48d5-9aaa-a0de4a5f3a25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:58.839092 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d873bf2-a5a2-48d5-9aaa-a0de4a5f3a25 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:58.845280 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 584/1161] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:31:58 2022] GET /identity/v3/role_assignments?user.id=277e8f2ab107459a880f3963f626cc6e&scope.project.id=0ecd3871ccd44830b009fd32136601ae&role.id=51b2d10dfffb4b75b186f3d8b15e2a78 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:32:00.837570 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39796ccb-8aa9-42d5-a16c-34b4df014696 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:00.838128 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39796ccb-8aa9-42d5-a16c-34b4df014696 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:00.838128 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39796ccb-8aa9-42d5-a16c-34b4df014696 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:00.839522 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 578/1162] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:32:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:32:00.844691 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb2266-3cdb-496e-a218-f969e6362aed None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:00.845080 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb2266-3cdb-496e-a218-f969e6362aed None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:00.845257 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb2266-3cdb-496e-a218-f969e6362aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:00.863388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-85eb2266-3cdb-496e-a218-f969e6362aed None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:32:00.864713 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-85eb2266-3cdb-496e-a218-f969e6362aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:32:00.881289 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:32:00.929186 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 585/1163] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:32:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:32:00.951608 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39796ccb-8aa9-42d5-a16c-34b4df014696 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:32:01.050935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39796ccb-8aa9-42d5-a16c-34b4df014696 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:32:01.083871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3308ffc6-a3a5-4341-adad-56a9b5db969c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:32:01.084866 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308ffc6-a3a5-4341-adad-56a9b5db969c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:01.085389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308ffc6-a3a5-4341-adad-56a9b5db969c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:01.085872 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308ffc6-a3a5-4341-adad-56a9b5db969c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:01.127652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:32:01.152800 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3308ffc6-a3a5-4341-adad-56a9b5db969c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:32:01.154340 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3308ffc6-a3a5-4341-adad-56a9b5db969c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:32:01.249539 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 579/1164] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:32:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:32:01.256150 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eb2266-3cdb-496e-a218-f969e6362aed None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:32:01.354769 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eb2266-3cdb-496e-a218-f969e6362aed None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:32:01.387711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9180eed-0271-48c4-9a92-94604e31157c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:32:01.388304 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9180eed-0271-48c4-9a92-94604e31157c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:01.388588 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9180eed-0271-48c4-9a92-94604e31157c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:01.388865 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9180eed-0271-48c4-9a92-94604e31157c None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:01.396045 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9180eed-0271-48c4-9a92-94604e31157c None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:32:01.397468 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9180eed-0271-48c4-9a92-94604e31157c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:32:01.408664 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-c9180eed-0271-48c4-9a92-94604e31157c None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 26 14:32:01.408664 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:32:01.408664 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 26 14:32:01.408664 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 26 14:32:01.408664 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:32:01.408664 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:32:01.408664 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:32:01.408664 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:32:01.408664 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 26 14:32:01.408664 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:32:01.408664 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:32:01.408664 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:32:01.408664 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:32:01.408664 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:32:01.408664 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:32:01.408664 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:32:01.408664 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:32:01.409834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:32:01.409834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:32:01.409834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:32:01.409834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:32:01.409834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:32:01.409834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:32:01.409834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:32:01.409834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:32:01.409834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:32:01.409834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:32:01.409834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:32:01.409834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:32:01.409834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:32:01.409834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:32:01.409834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:32:01.409834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:32:01.410680 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:32:01.410680 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:32:01.410680 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:32:01.410680 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:32:01.410680 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:32:01.410680 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:32:01.410680 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:32:01.410680 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:32:01.410680 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:32:01.410680 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:32:01.410680 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 26 14:32:01.410680 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:32:01.410680 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 26 14:32:01.410680 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:32:01.410680 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 586/1165] 10.222.0.97 () {56 vars in 1117 bytes} [Tue Jul 26 14:32:01 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:32:01.415925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3308ffc6-a3a5-4341-adad-56a9b5db969c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:32:01.488962 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:32:01.517264 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3308ffc6-a3a5-4341-adad-56a9b5db969c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:32:01.551255 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6855a24-01d1-4fb1-b3b1-8c656227420d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:32:01.551899 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6855a24-01d1-4fb1-b3b1-8c656227420d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:01.552187 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6855a24-01d1-4fb1-b3b1-8c656227420d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:01.552466 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6855a24-01d1-4fb1-b3b1-8c656227420d None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:01.553197 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6855a24-01d1-4fb1-b3b1-8c656227420d None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:32:01.553508 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6855a24-01d1-4fb1-b3b1-8c656227420d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:32:01.554872 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6855a24-01d1-4fb1-b3b1-8c656227420d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:32:01.562001 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 580/1166] 10.222.0.97 () {56 vars in 1115 bytes} [Tue Jul 26 14:32:01 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:32:03.530222 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad184aa-396b-4f14-a0df-eb85b1a91f8f None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:03.530733 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad184aa-396b-4f14-a0df-eb85b1a91f8f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:03.530822 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad184aa-396b-4f14-a0df-eb85b1a91f8f None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:03.532221 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 587/1167] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:32:03.537275 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c95611-dc7b-4b6a-96ad-5922549a9dac None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:03.537666 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c95611-dc7b-4b6a-96ad-5922549a9dac None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:03.537837 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c95611-dc7b-4b6a-96ad-5922549a9dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:03.555421 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-50c95611-dc7b-4b6a-96ad-5922549a9dac None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:32:03.556631 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-50c95611-dc7b-4b6a-96ad-5922549a9dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:32:03.620983 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 581/1168] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:32:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:32:03.644520 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad184aa-396b-4f14-a0df-eb85b1a91f8f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:32:03.659614 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:32:03.745191 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad184aa-396b-4f14-a0df-eb85b1a91f8f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:32:03.779039 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bce613-5b0c-48b0-934b-c8769bc84119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:32:03.779664 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bce613-5b0c-48b0-934b-c8769bc84119 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:03.779945 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bce613-5b0c-48b0-934b-c8769bc84119 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:03.780218 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bce613-5b0c-48b0-934b-c8769bc84119 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:03.781007 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bce613-5b0c-48b0-934b-c8769bc84119 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:32:03.782327 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bce613-5b0c-48b0-934b-c8769bc84119 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:32:03.874426 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 588/1169] 10.222.0.97 () {60 vars in 1138 bytes} [Tue Jul 26 14:32:03 2022] POST /identity/v3/domains => generated 267 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:32:05.897013 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed495e7e-b8c4-43c9-ab0d-e5c477031b1f None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:05.897541 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed495e7e-b8c4-43c9-ab0d-e5c477031b1f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:05.897541 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed495e7e-b8c4-43c9-ab0d-e5c477031b1f None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:05.898931 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 582/1170] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:32:05.904232 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2d201-8157-4b7c-8a5b-dbae06929636 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:05.904637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2d201-8157-4b7c-8a5b-dbae06929636 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:05.904785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2d201-8157-4b7c-8a5b-dbae06929636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:05.923507 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-28d2d201-8157-4b7c-8a5b-dbae06929636 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:32:05.924673 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-28d2d201-8157-4b7c-8a5b-dbae06929636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:32:05.989829 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 589/1171] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:32:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:32:06.012945 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed495e7e-b8c4-43c9-ab0d-e5c477031b1f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:32:06.078970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:32:06.114097 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed495e7e-b8c4-43c9-ab0d-e5c477031b1f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:32:06.147237 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136b0dc-46b6-4441-bf15-b96c983d126e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:32:06.147849 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136b0dc-46b6-4441-bf15-b96c983d126e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:06.148132 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136b0dc-46b6-4441-bf15-b96c983d126e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:06.148411 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136b0dc-46b6-4441-bf15-b96c983d126e None admin] PATH_INFO: `/v3/domains/e81ccdcf75394201b99682ded3cf705a` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:06.158639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c136b0dc-46b6-4441-bf15-b96c983d126e None admin] RBAC: Authorizing `identity:get_domain(domain_id=e81ccdcf75394201b99682ded3cf705a)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:32:06.160123 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c136b0dc-46b6-4441-bf15-b96c983d126e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:32:06.161723 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 583/1172] 10.222.0.97 () {56 vars in 1183 bytes} [Tue Jul 26 14:32:06 2022] GET /identity/v3/domains/e81ccdcf75394201b99682ded3cf705a => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:32:06.167626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d2d201-8157-4b7c-8a5b-dbae06929636 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:32:06.169787 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:32:06.263481 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d2d201-8157-4b7c-8a5b-dbae06929636 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:32:06.294974 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704840eb-937d-4c70-81a8-c9a57ca1777a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:32:06.295598 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704840eb-937d-4c70-81a8-c9a57ca1777a None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:06.295879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704840eb-937d-4c70-81a8-c9a57ca1777a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:06.296150 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704840eb-937d-4c70-81a8-c9a57ca1777a None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:06.297366 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704840eb-937d-4c70-81a8-c9a57ca1777a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:32:06.298769 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704840eb-937d-4c70-81a8-c9a57ca1777a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:32:06.405966 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 590/1173] 10.222.0.97 () {60 vars in 1141 bytes} [Tue Jul 26 14:32:06 2022] POST /identity/v3/projects => generated 361 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:32:08.419622 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba173031-fb94-4f42-bd4d-352a98c5bf3b None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:08.420162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba173031-fb94-4f42-bd4d-352a98c5bf3b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:08.420162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba173031-fb94-4f42-bd4d-352a98c5bf3b None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:08.421982 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 584/1174] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:32:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:32:08.427197 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b61f3c-754a-4c27-af9a-3a8dd7b937ff None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:08.427632 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b61f3c-754a-4c27-af9a-3a8dd7b937ff None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:08.427777 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b61f3c-754a-4c27-af9a-3a8dd7b937ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:08.446081 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-12b61f3c-754a-4c27-af9a-3a8dd7b937ff None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:32:08.447253 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-12b61f3c-754a-4c27-af9a-3a8dd7b937ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:32:08.495325 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:32:08.512339 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 591/1175] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:32:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:32:08.538024 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba173031-fb94-4f42-bd4d-352a98c5bf3b None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:32:08.637946 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba173031-fb94-4f42-bd4d-352a98c5bf3b None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:32:08.663293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:32:08.672221 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c4125c-ec4c-42af-ae41-f676a1db5954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:32:08.672820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4125c-ec4c-42af-ae41-f676a1db5954 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:08.673110 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4125c-ec4c-42af-ae41-f676a1db5954 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:08.673388 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4125c-ec4c-42af-ae41-f676a1db5954 None admin] PATH_INFO: `/v3/domains/e81ccdcf75394201b99682ded3cf705a` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:08.674549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c4125c-ec4c-42af-ae41-f676a1db5954 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e81ccdcf75394201b99682ded3cf705a)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:32:08.676020 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c4125c-ec4c-42af-ae41-f676a1db5954 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:32:08.677603 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 585/1176] 10.222.0.97 () {56 vars in 1183 bytes} [Tue Jul 26 14:32:08 2022] GET /identity/v3/domains/e81ccdcf75394201b99682ded3cf705a => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:32:08.683515 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b61f3c-754a-4c27-af9a-3a8dd7b937ff None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:32:08.778952 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b61f3c-754a-4c27-af9a-3a8dd7b937ff None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:32:08.812407 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a5b073-415c-45b8-835e-6f7f21a2853f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:32:08.813029 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5b073-415c-45b8-835e-6f7f21a2853f None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:08.813311 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5b073-415c-45b8-835e-6f7f21a2853f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:08.813588 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5b073-415c-45b8-835e-6f7f21a2853f None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:08.814259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a5b073-415c-45b8-835e-6f7f21a2853f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:32:08.815701 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a5b073-415c-45b8-835e-6f7f21a2853f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:32:08.919199 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 592/1177] 10.222.0.97 () {60 vars in 1132 bytes} [Tue Jul 26 14:32:08 2022] POST /identity/v3/users => generated 311 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:32:10.941540 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ef3af-8024-41a7-ad13-c696afd22077 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:10.942063 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ef3af-8024-41a7-ad13-c696afd22077 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:10.942177 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ef3af-8024-41a7-ad13-c696afd22077 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:10.943481 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 586/1178] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:32:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:32:10.948631 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c1eb6-86f6-4a20-9f01-3a3027d1a7a8 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:10.949014 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c1eb6-86f6-4a20-9f01-3a3027d1a7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:10.949187 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c1eb6-86f6-4a20-9f01-3a3027d1a7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:10.966830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-dc2c1eb6-86f6-4a20-9f01-3a3027d1a7a8 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:32:10.968059 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-dc2c1eb6-86f6-4a20-9f01-3a3027d1a7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:32:11.001724 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:32:11.034091 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 593/1179] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:32:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:32:11.057335 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ef3af-8024-41a7-ad13-c696afd22077 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:32:11.156802 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ef3af-8024-41a7-ad13-c696afd22077 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:32:11.189289 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aa843d-c768-4885-bb92-d461067aafbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:32:11.189915 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa843d-c768-4885-bb92-d461067aafbc None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:11.190196 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa843d-c768-4885-bb92-d461067aafbc None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:11.190467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa843d-c768-4885-bb92-d461067aafbc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:11.201030 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aa843d-c768-4885-bb92-d461067aafbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:32:11.205957 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-41aa843d-c768-4885-bb92-d461067aafbc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:32:11.205957 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:32:11.205957 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:32:11.205957 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:32:11.205957 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:32:11.205957 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:32:11.205957 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:32:11.205957 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:32:11.205957 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:32:11.205957 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:32:11.205957 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:32:11.205957 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:32:11.205957 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:32:11.205957 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:32:11.205957 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:32:11.205957 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:32:11.207066 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:32:11.207066 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:32:11.207066 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:32:11.207066 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:32:11.207066 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:32:11.207066 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:32:11.207066 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:32:11.207066 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:32:11.207066 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:32:11.207066 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:32:11.207066 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:32:11.207066 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:32:11.207066 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:32:11.207066 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:32:11.207066 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:32:11.207066 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:32:11.208180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:32:11.208180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:32:11.208180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:32:11.208180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:32:11.208180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:32:11.208180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:32:11.208180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:32:11.208180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:32:11.208180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:32:11.208180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:32:11.208180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:32:11.208180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:32:11.208180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 587/1180] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:32:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:32:11.213212 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c1eb6-86f6-4a20-9f01-3a3027d1a7a8 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:32:11.309999 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c1eb6-86f6-4a20-9f01-3a3027d1a7a8 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:32:11.343557 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97e746-1982-4c54-b0d1-e7d1a24dec8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:32:11.344180 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97e746-1982-4c54-b0d1-e7d1a24dec8e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:11.344464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97e746-1982-4c54-b0d1-e7d1a24dec8e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:11.344741 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97e746-1982-4c54-b0d1-e7d1a24dec8e None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:11.345456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97e746-1982-4c54-b0d1-e7d1a24dec8e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:32:11.345760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97e746-1982-4c54-b0d1-e7d1a24dec8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:32:11.347159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97e746-1982-4c54-b0d1-e7d1a24dec8e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:32:11.355045 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 594/1181] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:32:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:32:11.361352 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aa843d-c768-4885-bb92-d461067aafbc None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:32:11.415381 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:32:11.458874 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aa843d-c768-4885-bb92-d461067aafbc None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:32:11.491907 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1c877-6f93-4972-a5e5-85f0d8bd349d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:32:11.492530 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1c877-6f93-4972-a5e5-85f0d8bd349d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:11.492812 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1c877-6f93-4972-a5e5-85f0d8bd349d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:11.493087 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1c877-6f93-4972-a5e5-85f0d8bd349d None admin] PATH_INFO: `/v3/users/918d891e1dd24b048cceea4b2f0c1c5b` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:11.505736 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1c877-6f93-4972-a5e5-85f0d8bd349d None admin] RBAC: Authorizing `identity:get_user(user_id=918d891e1dd24b048cceea4b2f0c1c5b)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:32:11.507172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1c877-6f93-4972-a5e5-85f0d8bd349d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:32:11.508791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 588/1182] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:32:11 2022] GET /identity/v3/users/918d891e1dd24b048cceea4b2f0c1c5b => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:32:11.515278 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97e746-1982-4c54-b0d1-e7d1a24dec8e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:32:11.546736 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:32:11.614520 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97e746-1982-4c54-b0d1-e7d1a24dec8e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:32:11.647712 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d006a1ff-23cb-4c60-b167-86eadb2c6af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:32:11.648318 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006a1ff-23cb-4c60-b167-86eadb2c6af4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:11.648598 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006a1ff-23cb-4c60-b167-86eadb2c6af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:11.648868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006a1ff-23cb-4c60-b167-86eadb2c6af4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:11.713555 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d006a1ff-23cb-4c60-b167-86eadb2c6af4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:32:11.714911 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d006a1ff-23cb-4c60-b167-86eadb2c6af4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:32:11.743277 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:32:11.809503 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 595/1183] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:32:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:32:11.816078 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1c877-6f93-4972-a5e5-85f0d8bd349d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:32:11.915039 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1c877-6f93-4972-a5e5-85f0d8bd349d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:32:11.929165 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:32:11.950374 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6c1f2f-7499-4db1-9a6e-0733081cdbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:32:11.951024 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c1f2f-7499-4db1-9a6e-0733081cdbc8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:11.951308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c1f2f-7499-4db1-9a6e-0733081cdbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:11.951616 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c1f2f-7499-4db1-9a6e-0733081cdbc8 None admin] PATH_INFO: `/v3/projects/bf1fa8a9abb241769bada6d403aba024` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:11.953836 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6c1f2f-7499-4db1-9a6e-0733081cdbc8 None admin] RBAC: Authorizing `identity:get_project(project_id=bf1fa8a9abb241769bada6d403aba024)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:32:11.955338 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6c1f2f-7499-4db1-9a6e-0733081cdbc8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:32:11.957055 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 589/1184] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:32:11 2022] GET /identity/v3/projects/bf1fa8a9abb241769bada6d403aba024 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:32:11.963362 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d006a1ff-23cb-4c60-b167-86eadb2c6af4 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:32:12.060025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d006a1ff-23cb-4c60-b167-86eadb2c6af4 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:32:12.087850 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:32:12.092530 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df103c0-7fa6-463d-8b6e-4ce2da5ad647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:32:12.093674 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df103c0-7fa6-463d-8b6e-4ce2da5ad647 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:12.093952 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df103c0-7fa6-463d-8b6e-4ce2da5ad647 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:12.094222 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df103c0-7fa6-463d-8b6e-4ce2da5ad647 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:12.095019 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df103c0-7fa6-463d-8b6e-4ce2da5ad647 None admin] RBAC: Adding query filter params (role.id=51b2d10dfffb4b75b186f3d8b15e2a78, scope.project.id=bf1fa8a9abb241769bada6d403aba024, user.id=918d891e1dd24b048cceea4b2f0c1c5b) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:32:12.095345 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df103c0-7fa6-463d-8b6e-4ce2da5ad647 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:32:12.096803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df103c0-7fa6-463d-8b6e-4ce2da5ad647 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:32:12.101999 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 596/1185] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:32:11 2022] GET /identity/v3/role_assignments?user.id=918d891e1dd24b048cceea4b2f0c1c5b&scope.project.id=bf1fa8a9abb241769bada6d403aba024&role.id=51b2d10dfffb4b75b186f3d8b15e2a78 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:32:14.085468 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc689266-17df-487d-a49e-0581d07fe85d None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:14.085887 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc689266-17df-487d-a49e-0581d07fe85d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:14.086015 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc689266-17df-487d-a49e-0581d07fe85d None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:14.087382 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 590/1186] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:32:14.092767 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cf0b2-e335-4922-95e7-984556dbe256 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:14.093136 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cf0b2-e335-4922-95e7-984556dbe256 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:14.093334 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cf0b2-e335-4922-95e7-984556dbe256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:14.111023 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-215cf0b2-e335-4922-95e7-984556dbe256 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:32:14.112239 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-215cf0b2-e335-4922-95e7-984556dbe256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:32:14.176226 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 597/1187] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:32:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:32:14.198855 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc689266-17df-487d-a49e-0581d07fe85d None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:32:14.297864 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc689266-17df-487d-a49e-0581d07fe85d None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:32:14.331005 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a2d3d4-f1df-408b-8436-3643a45f5402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:32:14.331595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2d3d4-f1df-408b-8436-3643a45f5402 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:14.331877 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2d3d4-f1df-408b-8436-3643a45f5402 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:14.332150 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2d3d4-f1df-408b-8436-3643a45f5402 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:14.341914 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a2d3d4-f1df-408b-8436-3643a45f5402 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:32:14.346801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-38a2d3d4-f1df-408b-8436-3643a45f5402 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:32:14.346801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:32:14.346801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:32:14.346801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:32:14.346801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:32:14.346801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:32:14.346801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:32:14.346801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:32:14.346801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:32:14.346801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:32:14.346801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:32:14.346801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:32:14.346801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:32:14.346801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:32:14.346801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:32:14.346801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:32:14.347925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:32:14.347925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:32:14.347925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:32:14.347925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:32:14.347925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:32:14.347925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:32:14.347925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:32:14.347925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:32:14.347925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:32:14.347925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:32:14.347925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:32:14.347925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:32:14.347925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:32:14.347925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:32:14.347925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:32:14.347925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:32:14.348765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:32:14.348765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:32:14.348765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:32:14.348765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:32:14.348765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:32:14.348765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:32:14.348765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:32:14.348765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:32:14.348765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:32:14.348765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:32:14.348765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:32:14.348765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:32:14.348765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 591/1188] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:32:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:32:14.353859 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215cf0b2-e335-4922-95e7-984556dbe256 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:32:14.450252 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215cf0b2-e335-4922-95e7-984556dbe256 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:32:14.474714 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:32:14.483083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e51e80-4cdf-428b-96e7-3b2064460c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:32:14.484269 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e51e80-4cdf-428b-96e7-3b2064460c0d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:14.484610 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e51e80-4cdf-428b-96e7-3b2064460c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:14.484887 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e51e80-4cdf-428b-96e7-3b2064460c0d None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:14.485589 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e51e80-4cdf-428b-96e7-3b2064460c0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:32:14.485894 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e51e80-4cdf-428b-96e7-3b2064460c0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:32:14.487244 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e51e80-4cdf-428b-96e7-3b2064460c0d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:32:14.495172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 598/1189] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:32:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:32:14.501476 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a2d3d4-f1df-408b-8436-3643a45f5402 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:32:14.552601 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:32:14.611875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a2d3d4-f1df-408b-8436-3643a45f5402 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:32:14.645517 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d01a15-1382-4492-b7a7-013787a3bc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:32:14.646137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d01a15-1382-4492-b7a7-013787a3bc94 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:14.646413 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d01a15-1382-4492-b7a7-013787a3bc94 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:14.646681 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d01a15-1382-4492-b7a7-013787a3bc94 None admin] PATH_INFO: `/v3/users/918d891e1dd24b048cceea4b2f0c1c5b` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:14.647872 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d01a15-1382-4492-b7a7-013787a3bc94 None admin] RBAC: Authorizing `identity:get_user(user_id=918d891e1dd24b048cceea4b2f0c1c5b)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:32:14.649320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d01a15-1382-4492-b7a7-013787a3bc94 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:32:14.650920 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 592/1190] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:32:14 2022] GET /identity/v3/users/918d891e1dd24b048cceea4b2f0c1c5b => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:32:14.657217 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e51e80-4cdf-428b-96e7-3b2064460c0d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:32:14.755362 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e51e80-4cdf-428b-96e7-3b2064460c0d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:32:14.787736 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e358b8-2fe1-4b56-b7c6-1f3787cfe394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:32:14.788361 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e358b8-2fe1-4b56-b7c6-1f3787cfe394 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:14.788645 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e358b8-2fe1-4b56-b7c6-1f3787cfe394 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:14.788921 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e358b8-2fe1-4b56-b7c6-1f3787cfe394 None admin] PATH_INFO: `/v3/projects/bf1fa8a9abb241769bada6d403aba024` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:14.790054 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e358b8-2fe1-4b56-b7c6-1f3787cfe394 None admin] RBAC: Authorizing `identity:get_project(project_id=bf1fa8a9abb241769bada6d403aba024)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:32:14.791504 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e358b8-2fe1-4b56-b7c6-1f3787cfe394 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:32:14.793121 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 599/1191] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:32:14 2022] GET /identity/v3/projects/bf1fa8a9abb241769bada6d403aba024 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:32:14.799536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d01a15-1382-4492-b7a7-013787a3bc94 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:32:14.895823 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d01a15-1382-4492-b7a7-013787a3bc94 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:32:14.904732 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:32:14.929226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1605a9-5287-4d61-88c6-56f63d0564a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:32:14.930057 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1605a9-5287-4d61-88c6-56f63d0564a1 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:14.930335 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1605a9-5287-4d61-88c6-56f63d0564a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:14.930608 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1605a9-5287-4d61-88c6-56f63d0564a1 None admin] PATH_INFO: `/v3/projects/bf1fa8a9abb241769bada6d403aba024/users/918d891e1dd24b048cceea4b2f0c1c5b/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:14.932542 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1605a9-5287-4d61-88c6-56f63d0564a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=bf1fa8a9abb241769bada6d403aba024, user_id=918d891e1dd24b048cceea4b2f0c1c5b, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:32:14.933948 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1605a9-5287-4d61-88c6-56f63d0564a1 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:32:14.998272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 593/1192] 10.222.0.97 () {58 vars in 1439 bytes} [Tue Jul 26 14:32:14 2022] PUT /identity/v3/projects/bf1fa8a9abb241769bada6d403aba024/users/918d891e1dd24b048cceea4b2f0c1c5b/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:32:16.972644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4ec89-0feb-4ed3-92bc-16eb8ca3dc59 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:16.973172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4ec89-0feb-4ed3-92bc-16eb8ca3dc59 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:16.973172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4ec89-0feb-4ed3-92bc-16eb8ca3dc59 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:16.974530 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 600/1193] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:32:16.979598 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a67d3f-732d-402e-93f6-a5921de2346e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:16.979995 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a67d3f-732d-402e-93f6-a5921de2346e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:16.980152 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a67d3f-732d-402e-93f6-a5921de2346e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:16.998171 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-62a67d3f-732d-402e-93f6-a5921de2346e None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:32:16.999930 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-62a67d3f-732d-402e-93f6-a5921de2346e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:32:17.066395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 594/1194] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:32:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:32:17.089128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4ec89-0feb-4ed3-92bc-16eb8ca3dc59 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:32:17.146148 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:32:17.190265 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4ec89-0feb-4ed3-92bc-16eb8ca3dc59 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:32:17.223521 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c607fa-f9bb-427c-a7f1-c9dd377a2507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:32:17.224098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c607fa-f9bb-427c-a7f1-c9dd377a2507 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:17.224381 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c607fa-f9bb-427c-a7f1-c9dd377a2507 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:17.224656 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c607fa-f9bb-427c-a7f1-c9dd377a2507 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:17.235026 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c607fa-f9bb-427c-a7f1-c9dd377a2507 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:32:17.239946 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-b7c607fa-f9bb-427c-a7f1-c9dd377a2507 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:32:17.239946 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:32:17.239946 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:32:17.239946 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:32:17.239946 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:32:17.239946 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:32:17.239946 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:32:17.239946 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:32:17.239946 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:32:17.239946 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:32:17.239946 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:32:17.239946 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:32:17.239946 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:32:17.239946 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:32:17.239946 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:32:17.239946 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:32:17.241014 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:32:17.241014 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:32:17.241014 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:32:17.241014 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:32:17.241014 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:32:17.241014 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:32:17.241014 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:32:17.241014 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:32:17.241014 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:32:17.241014 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:32:17.241014 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:32:17.241014 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:32:17.241014 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:32:17.241014 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:32:17.241014 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:32:17.241014 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:32:17.242197 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:32:17.242197 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:32:17.242197 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:32:17.242197 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:32:17.242197 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:32:17.242197 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:32:17.242197 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:32:17.242197 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:32:17.242197 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:32:17.242197 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:32:17.242197 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:32:17.242197 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:32:17.242197 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 601/1195] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:32:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:32:17.247336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a67d3f-732d-402e-93f6-a5921de2346e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:32:17.344446 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:32:17.348084 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a67d3f-732d-402e-93f6-a5921de2346e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:32:17.381197 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1689d43-052d-45fc-8947-32299398fe27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:32:17.381845 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1689d43-052d-45fc-8947-32299398fe27 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:17.382165 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1689d43-052d-45fc-8947-32299398fe27 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:17.382478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1689d43-052d-45fc-8947-32299398fe27 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:17.383381 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1689d43-052d-45fc-8947-32299398fe27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:32:17.383763 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1689d43-052d-45fc-8947-32299398fe27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:32:17.385419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1689d43-052d-45fc-8947-32299398fe27 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:32:17.394439 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 595/1196] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:32:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:32:17.399874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c607fa-f9bb-427c-a7f1-c9dd377a2507 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:32:17.496893 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c607fa-f9bb-427c-a7f1-c9dd377a2507 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:32:17.512446 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:32:17.529959 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5517f14-6565-4d21-a57a-034215fb7e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:32:17.530596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5517f14-6565-4d21-a57a-034215fb7e99 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:17.530881 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5517f14-6565-4d21-a57a-034215fb7e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:17.531159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5517f14-6565-4d21-a57a-034215fb7e99 None admin] PATH_INFO: `/v3/users/918d891e1dd24b048cceea4b2f0c1c5b` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:17.532348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5517f14-6565-4d21-a57a-034215fb7e99 None admin] RBAC: Authorizing `identity:get_user(user_id=918d891e1dd24b048cceea4b2f0c1c5b)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:32:17.533813 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5517f14-6565-4d21-a57a-034215fb7e99 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:32:17.535386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 602/1197] 10.222.0.97 () {56 vars in 1177 bytes} [Tue Jul 26 14:32:17 2022] GET /identity/v3/users/918d891e1dd24b048cceea4b2f0c1c5b => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:32:17.541031 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1689d43-052d-45fc-8947-32299398fe27 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:32:17.637138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1689d43-052d-45fc-8947-32299398fe27 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:32:17.669363 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ed5e8-0cc2-4fbf-ae32-1947cf12ee80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:32:17.669967 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ed5e8-0cc2-4fbf-ae32-1947cf12ee80 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:17.670248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ed5e8-0cc2-4fbf-ae32-1947cf12ee80 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:17.671149 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ed5e8-0cc2-4fbf-ae32-1947cf12ee80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:17.698168 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:32:17.737055 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ed5e8-0cc2-4fbf-ae32-1947cf12ee80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:32:17.738497 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ed5e8-0cc2-4fbf-ae32-1947cf12ee80 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:32:17.834009 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 596/1198] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:32:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:32:17.839944 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5517f14-6565-4d21-a57a-034215fb7e99 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:32:17.938666 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5517f14-6565-4d21-a57a-034215fb7e99 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:32:17.971193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe23c1c-4adf-418a-bff1-44d30f13fb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:32:17.971858 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe23c1c-4adf-418a-bff1-44d30f13fb11 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:17.972146 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe23c1c-4adf-418a-bff1-44d30f13fb11 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:17.972425 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe23c1c-4adf-418a-bff1-44d30f13fb11 None admin] PATH_INFO: `/v3/projects/bf1fa8a9abb241769bada6d403aba024` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:17.973573 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe23c1c-4adf-418a-bff1-44d30f13fb11 None admin] RBAC: Authorizing `identity:get_project(project_id=bf1fa8a9abb241769bada6d403aba024)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:32:17.975011 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe23c1c-4adf-418a-bff1-44d30f13fb11 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:32:17.976692 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 603/1199] 10.222.0.97 () {56 vars in 1186 bytes} [Tue Jul 26 14:32:17 2022] GET /identity/v3/projects/bf1fa8a9abb241769bada6d403aba024 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:32:17.982821 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ed5e8-0cc2-4fbf-ae32-1947cf12ee80 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:32:18.040072 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:32:18.080595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ed5e8-0cc2-4fbf-ae32-1947cf12ee80 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:32:18.113892 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f9114-c766-4620-891c-aedd5a453e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:32:18.114447 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f9114-c766-4620-891c-aedd5a453e22 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:32:18.114732 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f9114-c766-4620-891c-aedd5a453e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:32:18.115011 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f9114-c766-4620-891c-aedd5a453e22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:32:18.115820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8f9114-c766-4620-891c-aedd5a453e22 None admin] RBAC: Adding query filter params (role.id=51b2d10dfffb4b75b186f3d8b15e2a78, scope.project.id=bf1fa8a9abb241769bada6d403aba024, user.id=918d891e1dd24b048cceea4b2f0c1c5b) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:32:18.116160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8f9114-c766-4620-891c-aedd5a453e22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:32:18.117581 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8f9114-c766-4620-891c-aedd5a453e22 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:32:18.123250 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 597/1200] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:32:17 2022] GET /identity/v3/role_assignments?user.id=918d891e1dd24b048cceea4b2f0c1c5b&scope.project.id=bf1fa8a9abb241769bada6d403aba024&role.id=51b2d10dfffb4b75b186f3d8b15e2a78 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:32:27.966633 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=74 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:32:28.120846 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=38 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:34.686661 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa7875-9baf-4e17-8632-033487f09a39 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:34.687255 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa7875-9baf-4e17-8632-033487f09a39 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:34.687255 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa7875-9baf-4e17-8632-033487f09a39 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:34.688657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 604/1201] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:33:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:33:34.693868 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9f802-4614-4777-83ae-e3d5c1637c32 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:34.694389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9f802-4614-4777-83ae-e3d5c1637c32 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:34.694485 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9f802-4614-4777-83ae-e3d5c1637c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:34.712697 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-f1d9f802-4614-4777-83ae-e3d5c1637c32 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:33:34.713887 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f1d9f802-4614-4777-83ae-e3d5c1637c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:33:34.781137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 598/1202] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:33:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:33:34.806793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa7875-9baf-4e17-8632-033487f09a39 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:34.907251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa7875-9baf-4e17-8632-033487f09a39 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:34.940665 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b5b23-6f14-4cc4-a637-966158437ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:34.941235 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b5b23-6f14-4cc4-a637-966158437ecd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:34.941523 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b5b23-6f14-4cc4-a637-966158437ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:34.941803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b5b23-6f14-4cc4-a637-966158437ecd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:34.952113 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89b5b23-6f14-4cc4-a637-966158437ecd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:34.955362 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89b5b23-6f14-4cc4-a637-966158437ecd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:34.967608 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-f89b5b23-6f14-4cc4-a637-966158437ecd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:33:34.967608 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:34.967608 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:34.967608 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:34.967608 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:34.967608 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:34.967608 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:34.967608 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:34.967608 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:34.967608 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:34.967608 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:34.967608 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:34.967608 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:33:34.967608 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:33:34.967608 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:33:34.968653 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:33:34.968653 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:34.968653 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:34.968653 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:34.968653 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:34.968653 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:34.968653 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:34.968653 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:34.968653 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:34.968653 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:34.968653 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:34.968653 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:34.968653 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:34.968653 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:34.968653 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:34.968653 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:34.968653 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:34.969641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:33:34.969641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:33:34.969641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:33:34.969641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:33:34.969641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 605/1203] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:33:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:33:34.974925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9f802-4614-4777-83ae-e3d5c1637c32 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:35.075073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9f802-4614-4777-83ae-e3d5c1637c32 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:35.104331 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:35.110091 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab721f-56c3-4b26-b11e-f9ca046e43df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:35.110711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab721f-56c3-4b26-b11e-f9ca046e43df None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:35.110993 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab721f-56c3-4b26-b11e-f9ca046e43df None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:35.111266 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab721f-56c3-4b26-b11e-f9ca046e43df None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:35.111985 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab721f-56c3-4b26-b11e-f9ca046e43df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:35.112290 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab721f-56c3-4b26-b11e-f9ca046e43df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:35.113652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab721f-56c3-4b26-b11e-f9ca046e43df None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:35.124851 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 599/1204] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:35.130948 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b5b23-6f14-4cc4-a637-966158437ecd None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:35.182627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:35.227826 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b5b23-6f14-4cc4-a637-966158437ecd None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:35.260100 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73aef8-7c49-4c99-bca8-7d3a711a0ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:35.260677 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73aef8-7c49-4c99-bca8-7d3a711a0ac6 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:35.261243 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73aef8-7c49-4c99-bca8-7d3a711a0ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:35.261743 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73aef8-7c49-4c99-bca8-7d3a711a0ac6 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:35.262978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73aef8-7c49-4c99-bca8-7d3a711a0ac6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:35.264450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73aef8-7c49-4c99-bca8-7d3a711a0ac6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:35.421567 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 606/1205] 10.222.0.97 () {60 vars in 1132 bytes} [Tue Jul 26 14:33:35 2022] POST /identity/v3/users => generated 251 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:33:37.477309 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9387a9-3325-41f1-8f60-6963beb1a0c1 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:37.477807 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9387a9-3325-41f1-8f60-6963beb1a0c1 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:37.477807 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9387a9-3325-41f1-8f60-6963beb1a0c1 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:37.479235 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 600/1206] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:33:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:33:37.484954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad47b41-3424-4f56-ac19-bdd6d45132ca None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:37.485361 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad47b41-3424-4f56-ac19-bdd6d45132ca None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:37.485600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad47b41-3424-4f56-ac19-bdd6d45132ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:37.506920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-2ad47b41-3424-4f56-ac19-bdd6d45132ca None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:33:37.508256 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2ad47b41-3424-4f56-ac19-bdd6d45132ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:33:37.580532 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 607/1207] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:33:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:33:37.604737 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9387a9-3325-41f1-8f60-6963beb1a0c1 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:37.706308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9387a9-3325-41f1-8f60-6963beb1a0c1 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:37.739467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a85a2e9-90f9-4cd8-804c-c202fe556d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:37.740174 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85a2e9-90f9-4cd8-804c-c202fe556d06 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:37.740507 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85a2e9-90f9-4cd8-804c-c202fe556d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:37.740937 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85a2e9-90f9-4cd8-804c-c202fe556d06 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:37.751715 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a85a2e9-90f9-4cd8-804c-c202fe556d06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:33:37.757461 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-6a85a2e9-90f9-4cd8-804c-c202fe556d06 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:33:37.757461 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:37.757461 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:37.757461 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:37.757461 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:37.757461 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:37.757461 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:37.757461 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:37.757461 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:37.757461 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:37.757461 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:37.757461 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:37.757461 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:33:37.757461 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:33:37.757461 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:33:37.757461 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:33:37.759244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:33:37.759244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:33:37.759244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:33:37.759244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:33:37.759244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:37.759244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:37.759244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:37.759244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:37.759244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:37.759244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:37.759244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:37.759244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:37.759244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:37.759244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:37.759244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:37.759244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:37.761273 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:37.761273 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:37.761273 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:37.761273 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:37.761273 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:33:37.761273 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:33:37.761273 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:33:37.761273 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:33:37.761273 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:33:37.761273 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:33:37.761273 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:33:37.761273 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:33:37.761273 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 601/1208] 10.222.0.97 () {56 vars in 1102 bytes} [Tue Jul 26 14:33:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:33:37.766432 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad47b41-3424-4f56-ac19-bdd6d45132ca None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:37.804956 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:37.861758 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad47b41-3424-4f56-ac19-bdd6d45132ca None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:37.894826 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17454c7b-2134-4765-82c4-d33bf1de67bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:37.895485 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17454c7b-2134-4765-82c4-d33bf1de67bc None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:37.895754 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17454c7b-2134-4765-82c4-d33bf1de67bc None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:37.896027 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17454c7b-2134-4765-82c4-d33bf1de67bc None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:37.896731 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17454c7b-2134-4765-82c4-d33bf1de67bc None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:37.897596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17454c7b-2134-4765-82c4-d33bf1de67bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:37.898959 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17454c7b-2134-4765-82c4-d33bf1de67bc None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:37.906752 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 608/1209] 10.222.0.97 () {56 vars in 1103 bytes} [Tue Jul 26 14:33:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:37.912367 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a85a2e9-90f9-4cd8-804c-c202fe556d06 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:37.962545 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:38.010584 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a85a2e9-90f9-4cd8-804c-c202fe556d06 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:38.042921 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db9695-0bc5-4c3c-85dd-c820c013e2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:38.043590 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db9695-0bc5-4c3c-85dd-c820c013e2ac None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:38.043879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db9695-0bc5-4c3c-85dd-c820c013e2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:38.044157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db9695-0bc5-4c3c-85dd-c820c013e2ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:38.055641 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13db9695-0bc5-4c3c-85dd-c820c013e2ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:38.057088 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13db9695-0bc5-4c3c-85dd-c820c013e2ac None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:38.071013 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-13db9695-0bc5-4c3c-85dd-c820c013e2ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:33:38.071013 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:38.071013 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:38.071013 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:38.071013 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:38.071013 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:38.071013 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:38.071013 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:38.071013 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:38.071013 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:38.071013 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:38.071013 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:38.071013 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:33:38.071013 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:33:38.071013 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:33:38.072073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:33:38.072073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:38.072073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:38.072073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:38.072073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:38.072073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:38.072073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:38.072073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:38.072073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:38.072073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:38.072073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:38.072073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:38.072073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:38.072073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:38.072073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:38.072073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:38.072073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:38.073167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:33:38.073167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:33:38.073167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:33:38.073167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:33:38.073167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 602/1210] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:33:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:33:38.077995 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17454c7b-2134-4765-82c4-d33bf1de67bc None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:38.176094 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:38.179007 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17454c7b-2134-4765-82c4-d33bf1de67bc None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:38.211784 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af46faae-adad-4462-992e-4d6f2869c4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:38.212394 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af46faae-adad-4462-992e-4d6f2869c4dc None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:38.212676 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af46faae-adad-4462-992e-4d6f2869c4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:38.212951 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af46faae-adad-4462-992e-4d6f2869c4dc None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:38.213632 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af46faae-adad-4462-992e-4d6f2869c4dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:38.213936 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af46faae-adad-4462-992e-4d6f2869c4dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:38.215315 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af46faae-adad-4462-992e-4d6f2869c4dc None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:38.226764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 609/1211] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:33:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:38.232469 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db9695-0bc5-4c3c-85dd-c820c013e2ac None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:38.335787 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db9695-0bc5-4c3c-85dd-c820c013e2ac None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:38.337307 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:38.368540 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3601c4-2cec-4594-85a3-f5115cf95f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:38.369296 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3601c4-2cec-4594-85a3-f5115cf95f98 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:38.369584 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3601c4-2cec-4594-85a3-f5115cf95f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:38.369900 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3601c4-2cec-4594-85a3-f5115cf95f98 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:38.377205 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3601c4-2cec-4594-85a3-f5115cf95f98 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:38.378632 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3601c4-2cec-4594-85a3-f5115cf95f98 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:38.390348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-9b3601c4-2cec-4594-85a3-f5115cf95f98 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 14:33:38.390348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:38.390348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:38.390348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:38.390348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:38.390348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:38.390348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:38.390348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:38.390348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:38.390348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:38.390348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:38.390348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:38.390348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:33:38.390348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:33:38.390348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:33:38.390348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:33:38.391585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:38.391585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:38.391585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:33:38.391585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:33:38.391585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:33:38.391585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:33:38.391585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:38.391585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:38.391585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:38.391585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:38.391585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:38.391585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:38.391585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:38.391585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:38.391585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:38.391585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:38.391585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:38.391585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:38.392775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:38.392775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:38.392775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:33:38.392775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:33:38.392775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:33:38.392775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:33:38.392775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:33:38.392775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:33:38.392775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 14:33:38.392775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:33:38.392775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 603/1212] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:33:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:33:38.397960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af46faae-adad-4462-992e-4d6f2869c4dc None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:38.494662 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af46faae-adad-4462-992e-4d6f2869c4dc None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:38.530075 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9910a4-c3c8-447d-bd67-f6f2a9e89fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:38.530690 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9910a4-c3c8-447d-bd67-f6f2a9e89fbb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:38.530975 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9910a4-c3c8-447d-bd67-f6f2a9e89fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:38.531254 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9910a4-c3c8-447d-bd67-f6f2a9e89fbb None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:38.532101 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9910a4-c3c8-447d-bd67-f6f2a9e89fbb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:38.532412 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9910a4-c3c8-447d-bd67-f6f2a9e89fbb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:38.533844 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9910a4-c3c8-447d-bd67-f6f2a9e89fbb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:38.550634 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 610/1213] 10.222.0.97 () {56 vars in 1143 bytes} [Tue Jul 26 14:33:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:38.556402 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3601c4-2cec-4594-85a3-f5115cf95f98 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:38.655067 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3601c4-2cec-4594-85a3-f5115cf95f98 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:38.688462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b558d9-2a1f-4df7-af27-1e525db18b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:38.689072 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b558d9-2a1f-4df7-af27-1e525db18b04 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:38.689350 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b558d9-2a1f-4df7-af27-1e525db18b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:38.689622 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b558d9-2a1f-4df7-af27-1e525db18b04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:38.719237 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:38.755687 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b558d9-2a1f-4df7-af27-1e525db18b04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:38.757209 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b558d9-2a1f-4df7-af27-1e525db18b04 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:38.855286 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 604/1214] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:33:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 14:33:38.861015 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9910a4-c3c8-447d-bd67-f6f2a9e89fbb None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:38.879829 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:38.959056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9910a4-c3c8-447d-bd67-f6f2a9e89fbb None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:38.993355 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da035a-04d2-4b0e-a8f1-9e115973a42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:38.993987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da035a-04d2-4b0e-a8f1-9e115973a42a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:38.994272 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da035a-04d2-4b0e-a8f1-9e115973a42a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:38.994546 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da035a-04d2-4b0e-a8f1-9e115973a42a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:39.004544 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da035a-04d2-4b0e-a8f1-9e115973a42a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:39.006322 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da035a-04d2-4b0e-a8f1-9e115973a42a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:39.021757 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-f7da035a-04d2-4b0e-a8f1-9e115973a42a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:33:39.021757 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:39.021757 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:39.021757 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:39.021757 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:39.021757 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:39.021757 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:39.021757 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:39.021757 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:39.021757 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:39.021757 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:39.021757 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:39.021757 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:33:39.021757 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:33:39.021757 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:33:39.025464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:33:39.025464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:39.025464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:39.025464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:39.025464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:39.025464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:39.025464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:39.025464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:39.025464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:39.025464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:39.025464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:39.025464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:39.025464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:39.025464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:39.025464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:39.025464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:39.025464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:39.027193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:33:39.027193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:33:39.027193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:33:39.027193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:33:39.027193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 611/1215] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:33:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:33:39.029466 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b558d9-2a1f-4df7-af27-1e525db18b04 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:39.092748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:39.130445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b558d9-2a1f-4df7-af27-1e525db18b04 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:39.166222 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7b1228-3d62-49e6-843d-09cd6fee7731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:39.166858 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b1228-3d62-49e6-843d-09cd6fee7731 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:39.167127 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b1228-3d62-49e6-843d-09cd6fee7731 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:39.167428 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b1228-3d62-49e6-843d-09cd6fee7731 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:39.168216 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7b1228-3d62-49e6-843d-09cd6fee7731 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:39.168457 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7b1228-3d62-49e6-843d-09cd6fee7731 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:39.169821 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7b1228-3d62-49e6-843d-09cd6fee7731 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:39.181160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 605/1216] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:39.186960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da035a-04d2-4b0e-a8f1-9e115973a42a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:39.254961 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:39.285133 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da035a-04d2-4b0e-a8f1-9e115973a42a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:39.319084 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de487d-959e-4013-bb04-a3b18dc192f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:39.319750 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de487d-959e-4013-bb04-a3b18dc192f4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:39.320031 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de487d-959e-4013-bb04-a3b18dc192f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:39.320303 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de487d-959e-4013-bb04-a3b18dc192f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:39.326705 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16de487d-959e-4013-bb04-a3b18dc192f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:39.328217 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16de487d-959e-4013-bb04-a3b18dc192f4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:39.340405 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-16de487d-959e-4013-bb04-a3b18dc192f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:33:39.340405 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:39.340405 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:39.340405 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:39.340405 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:39.340405 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:39.340405 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:39.340405 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:39.340405 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:39.340405 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:39.340405 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:39.340405 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:39.340405 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:33:39.340405 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:33:39.340405 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:33:39.341470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:33:39.341470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:39.341470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:39.341470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:39.341470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:39.341470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:39.341470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:39.341470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:39.341470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:39.341470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:39.341470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:39.341470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:39.341470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:39.341470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:39.341470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:39.341470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:39.341470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:39.342718 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:33:39.342718 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:33:39.342718 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:33:39.342718 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:33:39.342718 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:33:39.342718 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:33:39.342718 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:33:39.342718 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:33:39.342718 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 612/1217] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:33:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:33:39.347561 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7b1228-3d62-49e6-843d-09cd6fee7731 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:39.448228 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7b1228-3d62-49e6-843d-09cd6fee7731 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:39.470485 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:39.482415 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f82718-a597-43de-a446-6b7c0d8f1d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:39.483043 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f82718-a597-43de-a446-6b7c0d8f1d98 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:39.483326 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f82718-a597-43de-a446-6b7c0d8f1d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:39.483633 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f82718-a597-43de-a446-6b7c0d8f1d98 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:39.484383 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f82718-a597-43de-a446-6b7c0d8f1d98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:39.484691 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f82718-a597-43de-a446-6b7c0d8f1d98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:39.486121 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f82718-a597-43de-a446-6b7c0d8f1d98 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:39.498610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 606/1218] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:33:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:39.504689 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de487d-959e-4013-bb04-a3b18dc192f4 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:39.605537 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de487d-959e-4013-bb04-a3b18dc192f4 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:39.633071 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:39.638858 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db9b94-f4f4-4c9c-aaf6-ec733c2a8b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:39.639522 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db9b94-f4f4-4c9c-aaf6-ec733c2a8b09 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:39.639808 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db9b94-f4f4-4c9c-aaf6-ec733c2a8b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:39.640083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db9b94-f4f4-4c9c-aaf6-ec733c2a8b09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:39.640865 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21db9b94-f4f4-4c9c-aaf6-ec733c2a8b09 None admin] RBAC: Adding query filter params (role.id=9074ca9226f8483c885f32fa868a1acc, scope.project.id=c4c8da3011294783b2180ade1bfd9366, user.id=252c2d2939134416a3e8efb254f03c5b) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:39.641177 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21db9b94-f4f4-4c9c-aaf6-ec733c2a8b09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:39.642737 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21db9b94-f4f4-4c9c-aaf6-ec733c2a8b09 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:39.648580 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 613/1219] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:33:39 2022] GET /identity/v3/role_assignments?user.id=252c2d2939134416a3e8efb254f03c5b&scope.project.id=c4c8da3011294783b2180ade1bfd9366&role.id=9074ca9226f8483c885f32fa868a1acc => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:41.680103 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a31588-adbf-4bbb-8e70-948e8a8bf866 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:41.680649 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a31588-adbf-4bbb-8e70-948e8a8bf866 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:41.680649 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a31588-adbf-4bbb-8e70-948e8a8bf866 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:41.682086 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 607/1220] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:33:41.687275 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e892022-bab2-4907-8c45-cea74e1fd22f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:41.687757 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e892022-bab2-4907-8c45-cea74e1fd22f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:41.687877 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e892022-bab2-4907-8c45-cea74e1fd22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:41.706984 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-1e892022-bab2-4907-8c45-cea74e1fd22f None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:33:41.707907 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1e892022-bab2-4907-8c45-cea74e1fd22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:33:41.776593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 614/1221] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:33:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:33:41.800084 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a31588-adbf-4bbb-8e70-948e8a8bf866 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:41.908405 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a31588-adbf-4bbb-8e70-948e8a8bf866 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:41.942929 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef62e3-22fc-4f69-9d26-598224cc682b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:41.943532 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef62e3-22fc-4f69-9d26-598224cc682b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:41.943816 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef62e3-22fc-4f69-9d26-598224cc682b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:41.944092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef62e3-22fc-4f69-9d26-598224cc682b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:41.954592 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ef62e3-22fc-4f69-9d26-598224cc682b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:33:41.960059 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-02ef62e3-22fc-4f69-9d26-598224cc682b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:33:41.960059 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:41.960059 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:41.960059 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:41.960059 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:41.960059 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:41.960059 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:41.960059 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:41.960059 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:41.960059 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:41.960059 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:41.960059 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:41.960059 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:33:41.960059 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:33:41.960059 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:33:41.960059 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:33:41.961235 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:33:41.961235 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:33:41.961235 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:33:41.961235 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:33:41.961235 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:41.961235 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:41.961235 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:41.961235 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:41.961235 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:41.961235 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:41.961235 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:41.961235 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:41.961235 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:41.961235 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:41.961235 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:41.961235 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:41.962254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:41.962254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:41.962254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:41.962254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:41.962254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:33:41.962254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:33:41.962254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:33:41.962254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:33:41.962254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:33:41.962254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:33:41.962254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:33:41.962254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:33:41.962254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 608/1222] 10.222.0.97 () {56 vars in 1102 bytes} [Tue Jul 26 14:33:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:33:41.967505 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e892022-bab2-4907-8c45-cea74e1fd22f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:42.070560 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e892022-bab2-4907-8c45-cea74e1fd22f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:42.097447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:42.108878 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eee691-1591-45cb-8184-8e512bddbb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:42.109501 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eee691-1591-45cb-8184-8e512bddbb63 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:42.109784 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eee691-1591-45cb-8184-8e512bddbb63 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:42.110060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eee691-1591-45cb-8184-8e512bddbb63 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:42.110772 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eee691-1591-45cb-8184-8e512bddbb63 None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:42.111079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eee691-1591-45cb-8184-8e512bddbb63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:42.112497 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eee691-1591-45cb-8184-8e512bddbb63 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:42.120327 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 615/1223] 10.222.0.97 () {56 vars in 1103 bytes} [Tue Jul 26 14:33:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:42.126213 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef62e3-22fc-4f69-9d26-598224cc682b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:42.171569 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:42.228049 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef62e3-22fc-4f69-9d26-598224cc682b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:42.261937 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf07632-f515-4ba5-ac13-6edbcfb2a9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:42.262572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf07632-f515-4ba5-ac13-6edbcfb2a9f6 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:42.262854 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf07632-f515-4ba5-ac13-6edbcfb2a9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:42.263126 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf07632-f515-4ba5-ac13-6edbcfb2a9f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:42.272853 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf07632-f515-4ba5-ac13-6edbcfb2a9f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:42.274284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf07632-f515-4ba5-ac13-6edbcfb2a9f6 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:42.289053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-bbf07632-f515-4ba5-ac13-6edbcfb2a9f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:33:42.289053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:42.289053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:42.289053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:42.289053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:42.289053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:42.289053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:42.289053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:42.289053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:42.289053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:42.289053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:42.289053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:42.289053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:33:42.289053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:33:42.289053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:33:42.290120 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:33:42.290120 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:42.290120 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:42.290120 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:42.290120 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:42.290120 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:42.290120 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:42.290120 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:42.290120 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:42.290120 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:42.290120 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:42.290120 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:42.290120 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:42.290120 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:42.290120 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:42.290120 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:42.290120 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:42.291204 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:33:42.291204 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:33:42.291204 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:33:42.291204 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:33:42.291204 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 609/1224] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:33:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:33:42.296140 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eee691-1591-45cb-8184-8e512bddbb63 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:42.397713 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eee691-1591-45cb-8184-8e512bddbb63 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:42.429969 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089fdaf2-13f0-480a-98f9-be4d375b7b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:42.430588 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fdaf2-13f0-480a-98f9-be4d375b7b1e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:42.430876 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fdaf2-13f0-480a-98f9-be4d375b7b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:42.431155 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fdaf2-13f0-480a-98f9-be4d375b7b1e None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:42.431870 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089fdaf2-13f0-480a-98f9-be4d375b7b1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:42.432178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089fdaf2-13f0-480a-98f9-be4d375b7b1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:42.433556 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089fdaf2-13f0-480a-98f9-be4d375b7b1e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:42.446232 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 616/1225] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:42.452003 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf07632-f515-4ba5-ac13-6edbcfb2a9f6 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:42.544828 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:42.551904 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf07632-f515-4ba5-ac13-6edbcfb2a9f6 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:42.586113 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63177811-3e8b-4c73-83d7-1af60522b622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:42.586808 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63177811-3e8b-4c73-83d7-1af60522b622 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:42.587090 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63177811-3e8b-4c73-83d7-1af60522b622 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:42.587433 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63177811-3e8b-4c73-83d7-1af60522b622 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:42.594942 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63177811-3e8b-4c73-83d7-1af60522b622 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:42.596453 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63177811-3e8b-4c73-83d7-1af60522b622 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:42.608175 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-63177811-3e8b-4c73-83d7-1af60522b622 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 14:33:42.608175 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:42.608175 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:42.608175 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:42.608175 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:42.608175 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:42.608175 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:42.608175 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:42.608175 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:42.608175 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:42.608175 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:42.608175 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:42.608175 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:33:42.608175 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:33:42.608175 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:33:42.608175 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:33:42.609386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:42.609386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:42.609386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:33:42.609386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:33:42.609386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:33:42.609386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:33:42.609386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:42.609386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:42.609386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:42.609386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:42.609386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:42.609386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:42.609386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:42.609386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:42.609386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:42.609386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:42.609386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:42.609386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:42.610611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:42.610611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:42.610611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:33:42.610611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:33:42.610611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:33:42.610611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:33:42.610611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:33:42.610611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:33:42.610611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 14:33:42.610611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:33:42.610611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 610/1226] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:33:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:33:42.615142 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089fdaf2-13f0-480a-98f9-be4d375b7b1e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:42.662066 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:42.713447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089fdaf2-13f0-480a-98f9-be4d375b7b1e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:42.746898 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962a1ff-1a16-4c43-941c-d92ff0f4c1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:42.747611 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962a1ff-1a16-4c43-941c-d92ff0f4c1f0 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:42.747895 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962a1ff-1a16-4c43-941c-d92ff0f4c1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:42.748170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962a1ff-1a16-4c43-941c-d92ff0f4c1f0 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:42.748963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7962a1ff-1a16-4c43-941c-d92ff0f4c1f0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:42.749272 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7962a1ff-1a16-4c43-941c-d92ff0f4c1f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:42.750716 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7962a1ff-1a16-4c43-941c-d92ff0f4c1f0 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:42.767974 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 617/1227] 10.222.0.97 () {56 vars in 1143 bytes} [Tue Jul 26 14:33:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:42.774350 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63177811-3e8b-4c73-83d7-1af60522b622 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:42.874370 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63177811-3e8b-4c73-83d7-1af60522b622 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:42.908211 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9a3f5-2bbe-4d41-8354-cbb42bf9c839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:42.908837 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9a3f5-2bbe-4d41-8354-cbb42bf9c839 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:42.909114 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9a3f5-2bbe-4d41-8354-cbb42bf9c839 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:42.909385 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9a3f5-2bbe-4d41-8354-cbb42bf9c839 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:42.920342 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b9a3f5-2bbe-4d41-8354-cbb42bf9c839 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:42.921798 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b9a3f5-2bbe-4d41-8354-cbb42bf9c839 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:42.936181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-d7b9a3f5-2bbe-4d41-8354-cbb42bf9c839 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:33:42.936181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:42.936181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:42.936181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:42.936181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:42.936181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:42.936181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:42.936181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:42.936181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:42.936181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:42.936181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:42.936181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:42.936181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:33:42.936181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:33:42.936181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:33:42.942765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:33:42.942765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:42.942765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:42.942765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:42.942765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:42.942765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:42.942765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:42.942765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:42.942765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:42.942765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:42.942765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:42.942765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:42.942765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:42.942765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:42.942765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:42.942765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:42.942765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:42.943931 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:33:42.943931 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:33:42.943931 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:33:42.943931 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:33:42.943931 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 611/1228] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:33:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:33:42.944267 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962a1ff-1a16-4c43-941c-d92ff0f4c1f0 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:43.040528 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962a1ff-1a16-4c43-941c-d92ff0f4c1f0 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:43.042897 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:43.075828 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c782191-1ea0-4c82-bb60-e9a593d55209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:43.076474 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c782191-1ea0-4c82-bb60-e9a593d55209 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:43.076762 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c782191-1ea0-4c82-bb60-e9a593d55209 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:43.077041 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c782191-1ea0-4c82-bb60-e9a593d55209 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:43.077733 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c782191-1ea0-4c82-bb60-e9a593d55209 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:43.078042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c782191-1ea0-4c82-bb60-e9a593d55209 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:43.079422 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c782191-1ea0-4c82-bb60-e9a593d55209 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:43.090964 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 618/1229] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:43.097021 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9a3f5-2bbe-4d41-8354-cbb42bf9c839 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:43.106971 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:43.197014 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9a3f5-2bbe-4d41-8354-cbb42bf9c839 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:43.230228 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9059957-7edf-43b5-bf22-ab60c222b7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:43.230862 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9059957-7edf-43b5-bf22-ab60c222b7bf None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:43.231145 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9059957-7edf-43b5-bf22-ab60c222b7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:43.231442 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9059957-7edf-43b5-bf22-ab60c222b7bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:43.238275 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9059957-7edf-43b5-bf22-ab60c222b7bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:43.239741 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9059957-7edf-43b5-bf22-ab60c222b7bf None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:43.251677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-a9059957-7edf-43b5-bf22-ab60c222b7bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:33:43.251677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:43.251677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:43.251677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:43.251677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:43.251677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:43.251677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:43.251677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:43.251677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:43.251677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:43.251677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:43.251677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:43.251677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:33:43.251677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:33:43.251677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:33:43.252714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:33:43.252714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:43.252714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:43.252714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:43.252714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:43.252714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:43.252714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:43.252714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:43.252714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:43.252714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:43.252714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:43.252714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:43.252714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:43.252714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:43.252714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:43.252714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:43.252714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:43.253819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:33:43.253819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:33:43.253819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:33:43.253819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:33:43.253819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:33:43.253819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:33:43.253819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:33:43.253819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:33:43.253819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 612/1230] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:33:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:33:43.258993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c782191-1ea0-4c82-bb60-e9a593d55209 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:43.357825 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c782191-1ea0-4c82-bb60-e9a593d55209 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:43.391950 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7426d24-3d3a-4c6f-959b-3b2545f06569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:43.392550 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7426d24-3d3a-4c6f-959b-3b2545f06569 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:43.392831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7426d24-3d3a-4c6f-959b-3b2545f06569 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:43.393103 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7426d24-3d3a-4c6f-959b-3b2545f06569 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:43.393842 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7426d24-3d3a-4c6f-959b-3b2545f06569 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:43.394155 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7426d24-3d3a-4c6f-959b-3b2545f06569 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:43.395593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7426d24-3d3a-4c6f-959b-3b2545f06569 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:43.407840 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 619/1231] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:33:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:43.413905 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9059957-7edf-43b5-bf22-ab60c222b7bf None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:43.480172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:43.513194 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9059957-7edf-43b5-bf22-ab60c222b7bf None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:43.547085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c26910-48ae-4651-8db2-3b86034b6769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:43.547936 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c26910-48ae-4651-8db2-3b86034b6769 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:43.548241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c26910-48ae-4651-8db2-3b86034b6769 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:43.548524 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c26910-48ae-4651-8db2-3b86034b6769 None admin] PATH_INFO: `/v3/projects/c4c8da3011294783b2180ade1bfd9366/users/252c2d2939134416a3e8efb254f03c5b/roles/9074ca9226f8483c885f32fa868a1acc` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:43.562943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c26910-48ae-4651-8db2-3b86034b6769 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4c8da3011294783b2180ade1bfd9366, user_id=252c2d2939134416a3e8efb254f03c5b, role_id=9074ca9226f8483c885f32fa868a1acc)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:43.564875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c26910-48ae-4651-8db2-3b86034b6769 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:43.741696 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 613/1232] 10.222.0.97 () {58 vars in 1439 bytes} [Tue Jul 26 14:33:43 2022] PUT /identity/v3/projects/c4c8da3011294783b2180ade1bfd9366/users/252c2d2939134416a3e8efb254f03c5b/roles/9074ca9226f8483c885f32fa868a1acc => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:33:45.727957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69537a98-2272-4bbe-aa59-e31bc462d255 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:45.728487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69537a98-2272-4bbe-aa59-e31bc462d255 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:45.728487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69537a98-2272-4bbe-aa59-e31bc462d255 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:45.729927 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 620/1233] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:33:45.734989 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52007f82-470f-461f-8b59-7d5ccee1cf7d None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:45.738500 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52007f82-470f-461f-8b59-7d5ccee1cf7d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:45.738500 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52007f82-470f-461f-8b59-7d5ccee1cf7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:45.753899 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-52007f82-470f-461f-8b59-7d5ccee1cf7d None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:33:45.755346 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-52007f82-470f-461f-8b59-7d5ccee1cf7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:33:45.822105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 614/1234] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:33:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:33:45.845632 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69537a98-2272-4bbe-aa59-e31bc462d255 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:45.871332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:45.949804 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69537a98-2272-4bbe-aa59-e31bc462d255 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:45.984188 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a4c20-0b14-421f-a359-4a9c6f59681a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:45.984821 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a4c20-0b14-421f-a359-4a9c6f59681a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:45.985143 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a4c20-0b14-421f-a359-4a9c6f59681a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:45.985418 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a4c20-0b14-421f-a359-4a9c6f59681a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:45.996114 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a4c20-0b14-421f-a359-4a9c6f59681a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:33:46.000963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-460a4c20-0b14-421f-a359-4a9c6f59681a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:33:46.000963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:46.000963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:46.000963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:46.000963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:46.000963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:46.000963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:46.000963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:46.000963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:46.000963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:46.000963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:46.000963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:46.000963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:33:46.000963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:33:46.000963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:33:46.000963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:33:46.002916 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:33:46.002916 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:33:46.002916 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:33:46.002916 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:33:46.002916 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:46.002916 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:46.002916 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:46.002916 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:46.002916 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:46.002916 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:46.002916 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:46.002916 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:46.002916 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:46.002916 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:46.002916 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:46.002916 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:46.004062 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:46.004062 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:46.004062 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:46.004062 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:46.004062 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:33:46.004062 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:33:46.004062 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:33:46.004062 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:33:46.004062 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:33:46.004062 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:33:46.004062 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:33:46.004062 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:33:46.004062 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 621/1235] 10.222.0.97 () {56 vars in 1102 bytes} [Tue Jul 26 14:33:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:33:46.009101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52007f82-470f-461f-8b59-7d5ccee1cf7d None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:46.055885 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:46.107278 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52007f82-470f-461f-8b59-7d5ccee1cf7d None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:46.140901 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665e612-7d5f-4654-b95e-72a2691b161f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:46.141517 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665e612-7d5f-4654-b95e-72a2691b161f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:46.141800 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665e612-7d5f-4654-b95e-72a2691b161f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:46.142075 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665e612-7d5f-4654-b95e-72a2691b161f None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:46.142787 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665e612-7d5f-4654-b95e-72a2691b161f None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:46.143096 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665e612-7d5f-4654-b95e-72a2691b161f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:46.144492 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665e612-7d5f-4654-b95e-72a2691b161f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:46.152784 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 615/1236] 10.222.0.97 () {56 vars in 1103 bytes} [Tue Jul 26 14:33:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:46.158683 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a4c20-0b14-421f-a359-4a9c6f59681a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:46.243686 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:46.259416 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a4c20-0b14-421f-a359-4a9c6f59681a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:46.291787 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4160c6-4829-4789-90cd-ed1f8564d689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:46.292423 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4160c6-4829-4789-90cd-ed1f8564d689 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:46.292709 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4160c6-4829-4789-90cd-ed1f8564d689 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:46.292980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4160c6-4829-4789-90cd-ed1f8564d689 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:46.303441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4160c6-4829-4789-90cd-ed1f8564d689 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:46.304924 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4160c6-4829-4789-90cd-ed1f8564d689 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:46.320224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-0b4160c6-4829-4789-90cd-ed1f8564d689 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:33:46.320224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:46.320224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:46.320224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:46.320224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:46.320224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:46.320224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:46.320224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:46.320224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:46.320224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:46.320224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:46.320224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:46.320224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:33:46.320224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:33:46.320224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:33:46.321309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:33:46.321309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:46.321309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:46.321309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:46.321309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:46.321309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:46.321309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:46.321309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:46.321309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:46.321309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:46.321309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:46.321309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:46.321309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:46.321309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:46.321309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:46.321309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:46.321309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:46.322497 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:33:46.322497 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:33:46.322497 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:33:46.322497 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:33:46.322497 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 622/1237] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:33:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:33:46.327167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665e612-7d5f-4654-b95e-72a2691b161f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:46.436921 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665e612-7d5f-4654-b95e-72a2691b161f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:46.446123 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:46.470533 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7469aa-3f02-4698-a45f-2389780cdddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:46.471147 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7469aa-3f02-4698-a45f-2389780cdddc None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:46.471462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7469aa-3f02-4698-a45f-2389780cdddc None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:46.471742 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7469aa-3f02-4698-a45f-2389780cdddc None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:46.472430 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7469aa-3f02-4698-a45f-2389780cdddc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:46.472738 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7469aa-3f02-4698-a45f-2389780cdddc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:46.474105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7469aa-3f02-4698-a45f-2389780cdddc None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:46.486324 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 616/1238] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:46.492207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4160c6-4829-4789-90cd-ed1f8564d689 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:46.591564 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4160c6-4829-4789-90cd-ed1f8564d689 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:46.625336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8b476-1bc3-4dfb-b64f-7ec167160b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:46.625973 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8b476-1bc3-4dfb-b64f-7ec167160b98 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:46.626258 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8b476-1bc3-4dfb-b64f-7ec167160b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:46.626536 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8b476-1bc3-4dfb-b64f-7ec167160b98 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:46.629204 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:46.635118 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f8b476-1bc3-4dfb-b64f-7ec167160b98 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:46.636593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f8b476-1bc3-4dfb-b64f-7ec167160b98 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:46.648401 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-31f8b476-1bc3-4dfb-b64f-7ec167160b98 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 14:33:46.648401 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:46.648401 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:46.648401 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:46.648401 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:46.648401 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:46.648401 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:46.648401 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:46.648401 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:46.648401 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:46.648401 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:46.648401 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:46.648401 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:33:46.648401 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:33:46.648401 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:33:46.648401 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:33:46.649606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:46.649606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:46.649606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:33:46.649606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:33:46.649606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:33:46.649606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:33:46.649606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:46.649606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:46.649606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:46.649606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:46.649606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:46.649606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:46.649606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:46.649606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:46.649606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:46.649606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:46.649606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:46.649606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:46.650548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:46.650548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:46.650548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:33:46.650548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:33:46.650548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:33:46.650548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:33:46.650548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:33:46.650548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:33:46.650548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 14:33:46.650548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:33:46.650548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 623/1239] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:33:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:33:46.655601 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7469aa-3f02-4698-a45f-2389780cdddc None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:46.758221 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7469aa-3f02-4698-a45f-2389780cdddc None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:46.793907 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ea61b-7b79-4780-b442-2347e030e215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:46.794534 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ea61b-7b79-4780-b442-2347e030e215 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:46.794815 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ea61b-7b79-4780-b442-2347e030e215 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:46.795085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ea61b-7b79-4780-b442-2347e030e215 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:46.795910 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ea61b-7b79-4780-b442-2347e030e215 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:46.796259 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ea61b-7b79-4780-b442-2347e030e215 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:46.797680 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ea61b-7b79-4780-b442-2347e030e215 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:46.814607 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 617/1240] 10.222.0.97 () {56 vars in 1143 bytes} [Tue Jul 26 14:33:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:46.821062 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8b476-1bc3-4dfb-b64f-7ec167160b98 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:46.920894 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8b476-1bc3-4dfb-b64f-7ec167160b98 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:46.954780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05fa29c-b4e3-45fa-b0db-34bd11782825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:46.958993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fa29c-b4e3-45fa-b0db-34bd11782825 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:46.958993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fa29c-b4e3-45fa-b0db-34bd11782825 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:46.958993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fa29c-b4e3-45fa-b0db-34bd11782825 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:47.015795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:47.022124 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05fa29c-b4e3-45fa-b0db-34bd11782825 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:47.023590 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05fa29c-b4e3-45fa-b0db-34bd11782825 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:47.120017 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 624/1241] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:33:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:33:47.125799 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ea61b-7b79-4780-b442-2347e030e215 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:47.157761 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:47.230495 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ea61b-7b79-4780-b442-2347e030e215 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:47.265462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e422b-69f0-4653-aef5-892d99982658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:47.266094 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e422b-69f0-4653-aef5-892d99982658 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:47.266378 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e422b-69f0-4653-aef5-892d99982658 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:47.266656 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e422b-69f0-4653-aef5-892d99982658 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:47.276596 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e422b-69f0-4653-aef5-892d99982658 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:47.278031 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e422b-69f0-4653-aef5-892d99982658 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:47.292735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-032e422b-69f0-4653-aef5-892d99982658 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:33:47.292735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:47.292735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:47.292735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:47.292735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:47.292735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:47.292735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:47.292735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:47.292735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:47.292735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:47.292735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:47.292735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:47.292735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:33:47.292735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:33:47.292735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:33:47.293998 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:33:47.293998 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:47.293998 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:47.293998 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:47.293998 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:47.293998 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:47.293998 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:47.293998 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:47.293998 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:47.293998 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:47.293998 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:47.293998 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:47.293998 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:47.293998 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:47.293998 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:47.293998 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:47.293998 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:47.295036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:33:47.295036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:33:47.295036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:33:47.295036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:33:47.295036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 618/1242] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:33:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:33:47.299699 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05fa29c-b4e3-45fa-b0db-34bd11782825 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:47.393041 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:47.398669 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05fa29c-b4e3-45fa-b0db-34bd11782825 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:47.432494 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402514d-c170-4f54-a8bd-66db148671ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:47.433101 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402514d-c170-4f54-a8bd-66db148671ac None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:47.433383 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402514d-c170-4f54-a8bd-66db148671ac None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:47.433660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402514d-c170-4f54-a8bd-66db148671ac None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:47.434334 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b402514d-c170-4f54-a8bd-66db148671ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:47.434637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b402514d-c170-4f54-a8bd-66db148671ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:47.436043 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b402514d-c170-4f54-a8bd-66db148671ac None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:47.446766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 625/1243] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:33:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:47.452421 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e422b-69f0-4653-aef5-892d99982658 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:47.530971 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:47.553274 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e422b-69f0-4653-aef5-892d99982658 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:47.585892 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120e03c-0200-4afb-b5ae-9b142dce66e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:47.586529 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120e03c-0200-4afb-b5ae-9b142dce66e4 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:47.586811 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120e03c-0200-4afb-b5ae-9b142dce66e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:47.587087 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120e03c-0200-4afb-b5ae-9b142dce66e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:47.593645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6120e03c-0200-4afb-b5ae-9b142dce66e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:47.595018 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6120e03c-0200-4afb-b5ae-9b142dce66e4 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:47.606610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-6120e03c-0200-4afb-b5ae-9b142dce66e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:33:47.606610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:47.606610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:47.606610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:47.606610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:47.606610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:47.606610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:47.606610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:47.606610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:47.606610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:47.606610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:47.606610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:47.606610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:33:47.606610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:33:47.606610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:33:47.607768 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:33:47.607768 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:47.607768 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:47.607768 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:47.607768 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:47.607768 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:47.607768 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:47.607768 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:47.607768 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:47.607768 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:47.607768 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:47.607768 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:47.607768 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:47.607768 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:47.607768 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:47.607768 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:47.607768 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:47.613212 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:33:47.613212 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:33:47.613212 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:33:47.613212 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:33:47.613212 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:33:47.613212 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:33:47.613212 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:33:47.613212 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:33:47.613212 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 619/1244] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:33:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:33:47.615741 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402514d-c170-4f54-a8bd-66db148671ac None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:47.713346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402514d-c170-4f54-a8bd-66db148671ac None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:47.748123 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2c5dc-e44a-4e6e-aa77-54f9477ba9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:47.748740 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2c5dc-e44a-4e6e-aa77-54f9477ba9ef None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:47.749024 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2c5dc-e44a-4e6e-aa77-54f9477ba9ef None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:47.749299 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2c5dc-e44a-4e6e-aa77-54f9477ba9ef None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:47.750040 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2c5dc-e44a-4e6e-aa77-54f9477ba9ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:47.750343 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2c5dc-e44a-4e6e-aa77-54f9477ba9ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:47.751781 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2c5dc-e44a-4e6e-aa77-54f9477ba9ef None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:47.763870 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 626/1245] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:33:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:47.770276 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120e03c-0200-4afb-b5ae-9b142dce66e4 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:47.871218 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120e03c-0200-4afb-b5ae-9b142dce66e4 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:47.905592 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6076af-b7fc-4630-8cb8-d98be8b900f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:47.906199 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6076af-b7fc-4630-8cb8-d98be8b900f3 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:47.906473 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6076af-b7fc-4630-8cb8-d98be8b900f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:47.906744 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6076af-b7fc-4630-8cb8-d98be8b900f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:47.907543 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6076af-b7fc-4630-8cb8-d98be8b900f3 None admin] RBAC: Adding query filter params (role.id=9074ca9226f8483c885f32fa868a1acc, scope.project.id=c4c8da3011294783b2180ade1bfd9366, user.id=252c2d2939134416a3e8efb254f03c5b) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:47.907850 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6076af-b7fc-4630-8cb8-d98be8b900f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:47.909395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6076af-b7fc-4630-8cb8-d98be8b900f3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:47.916390 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 620/1246] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:33:47 2022] GET /identity/v3/role_assignments?user.id=252c2d2939134416a3e8efb254f03c5b&scope.project.id=c4c8da3011294783b2180ade1bfd9366&role.id=9074ca9226f8483c885f32fa868a1acc => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:49.953848 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05b1d9-637b-420c-af49-c3f330e27504 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:49.954403 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05b1d9-637b-420c-af49-c3f330e27504 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:49.954403 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05b1d9-637b-420c-af49-c3f330e27504 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:49.955684 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 627/1247] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:33:49.961516 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e0541-bd8e-4d77-bd89-ec851a65dd84 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:49.961887 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e0541-bd8e-4d77-bd89-ec851a65dd84 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:49.962061 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e0541-bd8e-4d77-bd89-ec851a65dd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:49.968385 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:49.981175 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-576e0541-bd8e-4d77-bd89-ec851a65dd84 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:33:49.987133 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-576e0541-bd8e-4d77-bd89-ec851a65dd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:33:50.048877 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 621/1248] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:33:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:33:50.073082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a05b1d9-637b-420c-af49-c3f330e27504 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:50.091371 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:50.171124 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a05b1d9-637b-420c-af49-c3f330e27504 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:50.204912 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d262ae-d77b-419f-951f-6ed3b91ae3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:50.205482 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d262ae-d77b-419f-951f-6ed3b91ae3d0 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:50.205768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d262ae-d77b-419f-951f-6ed3b91ae3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:50.206043 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d262ae-d77b-419f-951f-6ed3b91ae3d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:50.215957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d262ae-d77b-419f-951f-6ed3b91ae3d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:33:50.220793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-79d262ae-d77b-419f-951f-6ed3b91ae3d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:33:50.220793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:50.220793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:50.220793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:50.220793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:50.220793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:50.220793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:50.220793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:50.220793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:50.220793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:50.220793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:50.220793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:50.220793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:33:50.220793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:33:50.220793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:33:50.220793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:33:50.222060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:33:50.222060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:33:50.222060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:33:50.222060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:33:50.222060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:50.222060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:50.222060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:50.222060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:50.222060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:50.222060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:50.222060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:50.222060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:50.222060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:50.222060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:50.222060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:50.222060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:50.223095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:50.223095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:50.223095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:50.223095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:50.223095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:33:50.223095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:33:50.223095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:33:50.223095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:33:50.223095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:33:50.223095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:33:50.223095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:33:50.223095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:33:50.223095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 628/1249] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:33:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:33:50.228100 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e0541-bd8e-4d77-bd89-ec851a65dd84 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:50.326108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e0541-bd8e-4d77-bd89-ec851a65dd84 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:50.358064 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:50.363077 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8027503-6fa7-4178-a137-1c76e7b51c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:50.363737 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8027503-6fa7-4178-a137-1c76e7b51c76 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:50.364017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8027503-6fa7-4178-a137-1c76e7b51c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:50.364289 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8027503-6fa7-4178-a137-1c76e7b51c76 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:50.365074 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8027503-6fa7-4178-a137-1c76e7b51c76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:50.365375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8027503-6fa7-4178-a137-1c76e7b51c76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:50.366756 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8027503-6fa7-4178-a137-1c76e7b51c76 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:50.374865 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 622/1250] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:33:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:50.381362 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d262ae-d77b-419f-951f-6ed3b91ae3d0 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:50.461231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:50.482889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d262ae-d77b-419f-951f-6ed3b91ae3d0 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:50.515520 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac800262-826e-4331-bf2d-94976d5f9848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:50.516156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac800262-826e-4331-bf2d-94976d5f9848 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:50.516442 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac800262-826e-4331-bf2d-94976d5f9848 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:50.516798 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac800262-826e-4331-bf2d-94976d5f9848 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:50.527860 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac800262-826e-4331-bf2d-94976d5f9848 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:50.529306 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac800262-826e-4331-bf2d-94976d5f9848 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:50.543477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-ac800262-826e-4331-bf2d-94976d5f9848 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:33:50.543477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:50.543477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:50.543477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:50.543477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:50.543477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:50.543477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:50.543477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:50.543477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:50.543477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:50.543477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:50.543477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:50.543477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:33:50.543477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:33:50.543477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:33:50.544578 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:33:50.544578 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:50.544578 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:50.544578 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:50.544578 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:50.544578 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:50.544578 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:50.544578 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:50.544578 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:50.544578 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:50.544578 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:50.544578 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:50.544578 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:50.544578 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:50.544578 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:50.544578 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:50.544578 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:50.545624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:33:50.545624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:33:50.545624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:33:50.545624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:33:50.545624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 629/1251] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:33:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:33:50.550444 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8027503-6fa7-4178-a137-1c76e7b51c76 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:50.651118 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8027503-6fa7-4178-a137-1c76e7b51c76 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:50.686352 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d89232-ee05-4c89-a20e-2b58c855c26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:50.686973 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d89232-ee05-4c89-a20e-2b58c855c26d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:50.687272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d89232-ee05-4c89-a20e-2b58c855c26d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:50.687577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d89232-ee05-4c89-a20e-2b58c855c26d None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:50.688272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d89232-ee05-4c89-a20e-2b58c855c26d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:50.688567 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d89232-ee05-4c89-a20e-2b58c855c26d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:50.690740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d89232-ee05-4c89-a20e-2b58c855c26d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:50.702544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 623/1252] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:50.708628 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac800262-826e-4331-bf2d-94976d5f9848 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:50.809898 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac800262-826e-4331-bf2d-94976d5f9848 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:50.836105 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:50.843897 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eec79b-505a-4b1a-ac95-46b58c97c3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:50.844536 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eec79b-505a-4b1a-ac95-46b58c97c3a3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:50.844822 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eec79b-505a-4b1a-ac95-46b58c97c3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:50.845136 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eec79b-505a-4b1a-ac95-46b58c97c3a3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:50.852696 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eec79b-505a-4b1a-ac95-46b58c97c3a3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:50.854233 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eec79b-505a-4b1a-ac95-46b58c97c3a3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:50.865830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-a0eec79b-505a-4b1a-ac95-46b58c97c3a3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 14:33:50.865830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:50.865830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:50.865830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:50.865830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:50.865830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:50.865830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:50.865830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:50.865830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:50.865830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:50.865830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:50.865830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:50.865830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:33:50.865830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:33:50.865830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:33:50.865830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:33:50.866978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:50.866978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:50.866978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:33:50.866978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:33:50.866978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:33:50.866978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:33:50.866978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:50.866978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:50.866978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:50.866978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:50.866978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:50.866978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:50.866978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:50.866978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:50.866978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:50.866978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:50.866978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:50.866978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:50.868022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:50.868022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:50.868022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:33:50.868022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:33:50.868022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:33:50.868022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:33:50.868022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:33:50.868022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:33:50.868022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 14:33:50.868022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:33:50.868022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 630/1253] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:33:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:33:50.872895 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d89232-ee05-4c89-a20e-2b58c855c26d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:50.929380 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:50.973074 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d89232-ee05-4c89-a20e-2b58c855c26d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:51.006784 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a448683-bfa3-4f00-9525-5f38646e8c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:51.007529 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a448683-bfa3-4f00-9525-5f38646e8c7a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:51.007832 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a448683-bfa3-4f00-9525-5f38646e8c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:51.008112 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a448683-bfa3-4f00-9525-5f38646e8c7a None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:51.008900 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a448683-bfa3-4f00-9525-5f38646e8c7a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:51.009210 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a448683-bfa3-4f00-9525-5f38646e8c7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:51.010636 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a448683-bfa3-4f00-9525-5f38646e8c7a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:51.029154 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 624/1254] 10.222.0.97 () {56 vars in 1143 bytes} [Tue Jul 26 14:33:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:51.034956 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eec79b-505a-4b1a-ac95-46b58c97c3a3 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:51.136833 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eec79b-505a-4b1a-ac95-46b58c97c3a3 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:51.174059 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939bb67c-3939-4915-8aac-2267ce1b96ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:51.174722 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bb67c-3939-4915-8aac-2267ce1b96ad None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:51.174960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bb67c-3939-4915-8aac-2267ce1b96ad None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:51.175234 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bb67c-3939-4915-8aac-2267ce1b96ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:51.229198 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:51.241369 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939bb67c-3939-4915-8aac-2267ce1b96ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:51.242969 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939bb67c-3939-4915-8aac-2267ce1b96ad None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:51.342227 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 631/1255] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:33:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:33:51.367945 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a448683-bfa3-4f00-9525-5f38646e8c7a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:51.453346 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a448683-bfa3-4f00-9525-5f38646e8c7a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:51.466083 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:51.486413 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55bd631-5d3a-440d-8bfe-68db01527f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:51.487054 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bd631-5d3a-440d-8bfe-68db01527f11 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:51.487338 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bd631-5d3a-440d-8bfe-68db01527f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:51.487646 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bd631-5d3a-440d-8bfe-68db01527f11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:51.497965 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55bd631-5d3a-440d-8bfe-68db01527f11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:51.499427 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55bd631-5d3a-440d-8bfe-68db01527f11 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:51.513997 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-f55bd631-5d3a-440d-8bfe-68db01527f11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:33:51.513997 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:51.513997 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:51.513997 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:51.513997 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:51.513997 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:51.513997 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:51.513997 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:51.513997 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:51.513997 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:51.513997 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:51.513997 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:51.513997 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:33:51.513997 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:33:51.513997 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:33:51.515126 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:33:51.515126 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:51.515126 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:51.515126 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:51.515126 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:51.515126 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:51.515126 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:51.515126 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:51.515126 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:51.515126 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:51.515126 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:51.515126 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:51.515126 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:51.515126 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:51.515126 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:51.515126 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:51.515126 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:51.516160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:33:51.516160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:33:51.516160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:33:51.516160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:33:51.516160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 625/1256] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:33:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:33:51.521588 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939bb67c-3939-4915-8aac-2267ce1b96ad None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:51.606627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:51.618846 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939bb67c-3939-4915-8aac-2267ce1b96ad None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:51.652372 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02480c83-8278-4f65-acfa-fdfdce7cc4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:51.652993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02480c83-8278-4f65-acfa-fdfdce7cc4b6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:51.653276 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02480c83-8278-4f65-acfa-fdfdce7cc4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:51.653549 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02480c83-8278-4f65-acfa-fdfdce7cc4b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:51.654225 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02480c83-8278-4f65-acfa-fdfdce7cc4b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:51.654526 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02480c83-8278-4f65-acfa-fdfdce7cc4b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:51.655911 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02480c83-8278-4f65-acfa-fdfdce7cc4b6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:51.667320 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 632/1257] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:51.675681 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55bd631-5d3a-440d-8bfe-68db01527f11 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:51.772463 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55bd631-5d3a-440d-8bfe-68db01527f11 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:51.805489 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272edd0-2dd3-4615-b66e-d6d9633bc5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:51.806126 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272edd0-2dd3-4615-b66e-d6d9633bc5fb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:51.806410 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272edd0-2dd3-4615-b66e-d6d9633bc5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:51.806684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272edd0-2dd3-4615-b66e-d6d9633bc5fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:51.814171 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7272edd0-2dd3-4615-b66e-d6d9633bc5fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:51.815633 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7272edd0-2dd3-4615-b66e-d6d9633bc5fb None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:51.826822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-7272edd0-2dd3-4615-b66e-d6d9633bc5fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:33:51.826822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:51.826822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:51.826822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:51.826822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:51.826822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:51.826822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:51.826822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:51.826822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:51.826822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:51.826822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:51.826822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:51.826822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:33:51.826822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:33:51.826822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:33:51.827888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:33:51.827888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:51.827888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:51.827888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:51.827888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:51.827888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:51.827888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:51.827888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:51.827888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:51.827888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:51.827888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:51.827888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:51.827888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:51.827888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:51.827888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:51.827888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:51.827888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:51.828987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:33:51.828987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:33:51.828987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:33:51.828987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:33:51.828987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:33:51.828987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:33:51.828987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:33:51.828987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:33:51.828987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 626/1258] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:33:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:33:51.834465 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02480c83-8278-4f65-acfa-fdfdce7cc4b6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:51.934006 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02480c83-8278-4f65-acfa-fdfdce7cc4b6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:51.967584 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3c0d4-5b5b-46b6-858f-ed0f4847a3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:51.968254 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3c0d4-5b5b-46b6-858f-ed0f4847a3df None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:51.968537 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3c0d4-5b5b-46b6-858f-ed0f4847a3df None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:51.968811 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3c0d4-5b5b-46b6-858f-ed0f4847a3df None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:51.969544 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3c0d4-5b5b-46b6-858f-ed0f4847a3df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:51.969850 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3c0d4-5b5b-46b6-858f-ed0f4847a3df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:51.971288 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3c0d4-5b5b-46b6-858f-ed0f4847a3df None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:51.983410 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 633/1259] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:33:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:51.989663 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272edd0-2dd3-4615-b66e-d6d9633bc5fb None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:52.015867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:52.089340 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272edd0-2dd3-4615-b66e-d6d9633bc5fb None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:52.123541 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668f289a-847f-4638-9059-5c3f72f9c275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:52.124153 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f289a-847f-4638-9059-5c3f72f9c275 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:52.124459 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f289a-847f-4638-9059-5c3f72f9c275 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:52.124733 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f289a-847f-4638-9059-5c3f72f9c275 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:52.125575 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668f289a-847f-4638-9059-5c3f72f9c275 None admin] RBAC: Adding query filter params (role.id=51b2d10dfffb4b75b186f3d8b15e2a78, scope.project.id=c4c8da3011294783b2180ade1bfd9366, user.id=252c2d2939134416a3e8efb254f03c5b) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:52.125937 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668f289a-847f-4638-9059-5c3f72f9c275 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:52.127600 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668f289a-847f-4638-9059-5c3f72f9c275 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:52.133883 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 627/1260] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:33:51 2022] GET /identity/v3/role_assignments?user.id=252c2d2939134416a3e8efb254f03c5b&scope.project.id=c4c8da3011294783b2180ade1bfd9366&role.id=51b2d10dfffb4b75b186f3d8b15e2a78 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:54.139809 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bacd09-57e0-4340-9d17-2ee09c06351d None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:54.140342 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bacd09-57e0-4340-9d17-2ee09c06351d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:54.140342 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bacd09-57e0-4340-9d17-2ee09c06351d None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:54.141717 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 634/1261] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:33:54.146738 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a773fbd-5cc3-4b66-bd3a-fef5b997adaf None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:54.147158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a773fbd-5cc3-4b66-bd3a-fef5b997adaf None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:54.147298 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a773fbd-5cc3-4b66-bd3a-fef5b997adaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:54.166210 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-7a773fbd-5cc3-4b66-bd3a-fef5b997adaf None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:33:54.167395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7a773fbd-5cc3-4b66-bd3a-fef5b997adaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:33:54.233187 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 628/1262] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:33:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:33:54.256268 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bacd09-57e0-4340-9d17-2ee09c06351d None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:54.267119 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:54.356447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bacd09-57e0-4340-9d17-2ee09c06351d None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:54.391169 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056f65f-4ad4-48b9-b033-938de856e981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:54.391833 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056f65f-4ad4-48b9-b033-938de856e981 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:54.392119 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056f65f-4ad4-48b9-b033-938de856e981 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:54.392399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056f65f-4ad4-48b9-b033-938de856e981 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:54.403010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056f65f-4ad4-48b9-b033-938de856e981 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:33:54.408117 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-7056f65f-4ad4-48b9-b033-938de856e981 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:33:54.408117 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:54.408117 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:54.408117 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:54.408117 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:54.408117 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:54.408117 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:54.408117 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:54.408117 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:54.408117 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:54.408117 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:54.408117 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:54.408117 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:33:54.408117 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:33:54.408117 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:33:54.408117 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:33:54.409259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:33:54.409259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:33:54.409259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:33:54.409259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:33:54.409259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:54.409259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:54.409259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:54.409259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:54.409259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:54.409259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:54.409259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:54.409259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:54.409259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:54.409259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:54.409259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:54.409259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:54.410324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:54.410324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:54.410324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:54.410324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:54.410324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:33:54.410324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:33:54.410324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:33:54.410324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:33:54.410324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:33:54.410324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:33:54.410324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:33:54.410324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:33:54.410324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 635/1263] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:33:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:33:54.415856 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a773fbd-5cc3-4b66-bd3a-fef5b997adaf None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:54.435137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:54.514323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a773fbd-5cc3-4b66-bd3a-fef5b997adaf None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:54.549842 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea428ee-dc7d-4a2c-a1f7-bcab43236b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:54.550461 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea428ee-dc7d-4a2c-a1f7-bcab43236b2a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:54.550747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea428ee-dc7d-4a2c-a1f7-bcab43236b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:54.551028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea428ee-dc7d-4a2c-a1f7-bcab43236b2a None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:54.551803 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea428ee-dc7d-4a2c-a1f7-bcab43236b2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:54.552115 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea428ee-dc7d-4a2c-a1f7-bcab43236b2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:54.553483 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea428ee-dc7d-4a2c-a1f7-bcab43236b2a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:54.561782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 629/1264] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:33:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:54.567667 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056f65f-4ad4-48b9-b033-938de856e981 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:54.638295 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:54.670646 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056f65f-4ad4-48b9-b033-938de856e981 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:54.703711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c848f80d-d6df-471c-bd5b-c1536ad5baad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:54.704342 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848f80d-d6df-471c-bd5b-c1536ad5baad None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:54.704624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848f80d-d6df-471c-bd5b-c1536ad5baad None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:54.704899 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848f80d-d6df-471c-bd5b-c1536ad5baad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:54.715159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c848f80d-d6df-471c-bd5b-c1536ad5baad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:54.716617 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c848f80d-d6df-471c-bd5b-c1536ad5baad None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:54.730287 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-c848f80d-d6df-471c-bd5b-c1536ad5baad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:33:54.730287 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:54.730287 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:54.730287 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:54.730287 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:54.730287 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:54.730287 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:54.730287 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:54.730287 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:54.730287 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:54.730287 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:54.730287 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:54.730287 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:33:54.730287 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:33:54.730287 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:33:54.731309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:33:54.731309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:54.731309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:54.731309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:54.731309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:54.731309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:54.731309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:54.731309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:54.731309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:54.731309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:54.731309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:54.731309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:54.731309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:54.731309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:54.731309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:54.731309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:54.731309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:54.732286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:33:54.732286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:33:54.732286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:33:54.732286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:33:54.732286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 636/1265] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:33:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:33:54.737366 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea428ee-dc7d-4a2c-a1f7-bcab43236b2a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:54.819246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:54.837505 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea428ee-dc7d-4a2c-a1f7-bcab43236b2a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:54.870476 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47e2b1-803a-4693-91c2-97449ac24ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:54.871081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47e2b1-803a-4693-91c2-97449ac24ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:54.871367 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47e2b1-803a-4693-91c2-97449ac24ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:54.871679 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47e2b1-803a-4693-91c2-97449ac24ec2 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:54.872364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a47e2b1-803a-4693-91c2-97449ac24ec2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:54.872666 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a47e2b1-803a-4693-91c2-97449ac24ec2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:54.874015 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a47e2b1-803a-4693-91c2-97449ac24ec2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:54.885846 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 630/1266] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:33:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:54.891630 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c848f80d-d6df-471c-bd5b-c1536ad5baad None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:54.992323 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c848f80d-d6df-471c-bd5b-c1536ad5baad None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:55.011859 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:55.026021 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1fa0b4-ac47-423d-b676-65092003e83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:55.026657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1fa0b4-ac47-423d-b676-65092003e83d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:55.026939 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1fa0b4-ac47-423d-b676-65092003e83d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:55.027217 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1fa0b4-ac47-423d-b676-65092003e83d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:55.034505 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1fa0b4-ac47-423d-b676-65092003e83d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:55.036001 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1fa0b4-ac47-423d-b676-65092003e83d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:55.047190 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-3f1fa0b4-ac47-423d-b676-65092003e83d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 14:33:55.047190 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:55.047190 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:55.047190 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:55.047190 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:55.047190 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:55.047190 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:55.047190 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:55.047190 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:55.047190 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:55.047190 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:55.047190 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:55.047190 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:33:55.047190 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:33:55.047190 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:33:55.047190 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:33:55.048491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:55.048491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:55.048491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:33:55.048491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:33:55.048491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:33:55.048491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:33:55.048491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:55.048491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:55.048491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:55.048491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:55.048491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:55.048491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:55.048491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:55.048491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:55.048491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:55.048491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:55.048491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:55.048491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:55.049677 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:55.049677 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:55.049677 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:33:55.049677 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:33:55.049677 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:33:55.049677 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:33:55.049677 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:33:55.049677 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:33:55.049677 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 14:33:55.049677 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:33:55.053064 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 637/1267] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:33:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:33:55.054106 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47e2b1-803a-4693-91c2-97449ac24ec2 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:55.152738 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47e2b1-803a-4693-91c2-97449ac24ec2 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:55.184987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc40b551-6511-48d5-b530-8bc04c8c756b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:55.191866 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40b551-6511-48d5-b530-8bc04c8c756b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:55.191866 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40b551-6511-48d5-b530-8bc04c8c756b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:55.191866 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40b551-6511-48d5-b530-8bc04c8c756b None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:55.191866 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc40b551-6511-48d5-b530-8bc04c8c756b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:55.191866 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc40b551-6511-48d5-b530-8bc04c8c756b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:55.191866 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc40b551-6511-48d5-b530-8bc04c8c756b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:55.198480 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:55.206688 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 631/1268] 10.222.0.97 () {56 vars in 1143 bytes} [Tue Jul 26 14:33:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:55.212987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1fa0b4-ac47-423d-b676-65092003e83d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:55.312035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1fa0b4-ac47-423d-b676-65092003e83d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:55.345929 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40058f-8032-42e7-8927-bcc0b47d39b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:55.346562 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40058f-8032-42e7-8927-bcc0b47d39b3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:55.346848 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40058f-8032-42e7-8927-bcc0b47d39b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:55.347126 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40058f-8032-42e7-8927-bcc0b47d39b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:55.357520 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e40058f-8032-42e7-8927-bcc0b47d39b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:55.360169 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e40058f-8032-42e7-8927-bcc0b47d39b3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:55.373317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-5e40058f-8032-42e7-8927-bcc0b47d39b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:33:55.373317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:55.373317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:55.373317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:55.373317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:55.373317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:55.373317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:55.373317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:55.373317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:55.373317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:55.373317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:55.373317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:55.373317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:33:55.373317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:33:55.373317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:33:55.374196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:33:55.374196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:55.374196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:55.374196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:55.374196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:55.374196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:55.374196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:55.374196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:55.374196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:55.374196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:55.374196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:55.374196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:55.374196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:55.374196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:55.374196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:55.374196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:55.374196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:55.375111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:33:55.375111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:33:55.375111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:33:55.375111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:33:55.375111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 638/1269] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:33:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:33:55.381209 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc40b551-6511-48d5-b530-8bc04c8c756b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:55.480158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc40b551-6511-48d5-b530-8bc04c8c756b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:55.513953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c421ed-4c3d-4cb8-8faa-ca91a3960141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:55.514569 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c421ed-4c3d-4cb8-8faa-ca91a3960141 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:55.514856 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c421ed-4c3d-4cb8-8faa-ca91a3960141 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:55.515141 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c421ed-4c3d-4cb8-8faa-ca91a3960141 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:55.515857 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c421ed-4c3d-4cb8-8faa-ca91a3960141 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:55.516163 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c421ed-4c3d-4cb8-8faa-ca91a3960141 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:55.517521 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c421ed-4c3d-4cb8-8faa-ca91a3960141 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:55.529198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 632/1270] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:55.535082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40058f-8032-42e7-8927-bcc0b47d39b3 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:55.566718 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:55.636314 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40058f-8032-42e7-8927-bcc0b47d39b3 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:55.669135 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a72af7-41ef-4c69-a61b-1d2fa8588012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:55.669795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a72af7-41ef-4c69-a61b-1d2fa8588012 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:55.670080 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a72af7-41ef-4c69-a61b-1d2fa8588012 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:55.670354 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a72af7-41ef-4c69-a61b-1d2fa8588012 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:55.676925 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a72af7-41ef-4c69-a61b-1d2fa8588012 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:55.678356 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a72af7-41ef-4c69-a61b-1d2fa8588012 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:55.689640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-21a72af7-41ef-4c69-a61b-1d2fa8588012 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:33:55.689640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:55.689640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:55.689640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:55.689640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:55.689640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:55.689640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:55.689640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:55.689640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:55.689640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:55.689640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:55.689640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:55.689640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:33:55.689640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:33:55.689640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:33:55.693700 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:33:55.693700 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:55.693700 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:55.693700 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:55.693700 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:55.693700 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:55.693700 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:55.693700 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:55.693700 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:55.693700 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:55.693700 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:55.693700 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:55.693700 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:55.693700 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:55.693700 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:55.693700 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:55.693700 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:55.694883 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:33:55.694883 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:33:55.694883 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:33:55.694883 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:33:55.694883 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:33:55.694883 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:33:55.694883 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:33:55.694883 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:33:55.694883 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 639/1271] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:33:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:33:55.697309 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c421ed-4c3d-4cb8-8faa-ca91a3960141 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:55.755036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:55.795778 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c421ed-4c3d-4cb8-8faa-ca91a3960141 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:55.830424 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053717e-3ea3-41e7-9a77-42f9d2a8b1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:55.833543 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053717e-3ea3-41e7-9a77-42f9d2a8b1c8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:55.833543 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053717e-3ea3-41e7-9a77-42f9d2a8b1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:55.833543 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053717e-3ea3-41e7-9a77-42f9d2a8b1c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:55.833543 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2053717e-3ea3-41e7-9a77-42f9d2a8b1c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:55.833543 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2053717e-3ea3-41e7-9a77-42f9d2a8b1c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:55.834650 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2053717e-3ea3-41e7-9a77-42f9d2a8b1c8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:55.847567 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 633/1272] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:33:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:55.853538 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a72af7-41ef-4c69-a61b-1d2fa8588012 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:55.944489 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:55.952874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a72af7-41ef-4c69-a61b-1d2fa8588012 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:55.984848 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eaf9e3-8cb4-46eb-8100-d4babaf22a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:55.985502 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eaf9e3-8cb4-46eb-8100-d4babaf22a4f None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:55.985785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eaf9e3-8cb4-46eb-8100-d4babaf22a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:55.986063 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eaf9e3-8cb4-46eb-8100-d4babaf22a4f None admin] PATH_INFO: `/v3/projects/c4c8da3011294783b2180ade1bfd9366/users/252c2d2939134416a3e8efb254f03c5b/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:55.987986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eaf9e3-8cb4-46eb-8100-d4babaf22a4f None admin] RBAC: Authorizing `identity:create_grant(project_id=c4c8da3011294783b2180ade1bfd9366, user_id=252c2d2939134416a3e8efb254f03c5b, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:55.990117 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eaf9e3-8cb4-46eb-8100-d4babaf22a4f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:56.122707 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 640/1273] 10.222.0.97 () {58 vars in 1439 bytes} [Tue Jul 26 14:33:55 2022] PUT /identity/v3/projects/c4c8da3011294783b2180ade1bfd9366/users/252c2d2939134416a3e8efb254f03c5b/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:33:58.116138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732182ab-e345-4557-bace-f981ba5223d8 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:58.116674 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732182ab-e345-4557-bace-f981ba5223d8 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:58.116674 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732182ab-e345-4557-bace-f981ba5223d8 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:58.118083 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 634/1274] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:33:58.123515 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacaadf-ca56-4dc5-8c24-620a3487d2c3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:58.123952 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacaadf-ca56-4dc5-8c24-620a3487d2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:58.124082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacaadf-ca56-4dc5-8c24-620a3487d2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:58.142440 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-2cacaadf-ca56-4dc5-8c24-620a3487d2c3 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:33:58.143808 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2cacaadf-ca56-4dc5-8c24-620a3487d2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:33:58.210948 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 641/1275] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:33:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:33:58.234218 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732182ab-e345-4557-bace-f981ba5223d8 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:58.337089 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732182ab-e345-4557-bace-f981ba5223d8 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:58.354762 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:58.371076 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8770002-165c-4d3e-97bf-cc0d45898e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:58.371749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8770002-165c-4d3e-97bf-cc0d45898e94 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:58.372033 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8770002-165c-4d3e-97bf-cc0d45898e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:58.372306 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8770002-165c-4d3e-97bf-cc0d45898e94 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:58.382905 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8770002-165c-4d3e-97bf-cc0d45898e94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:33:58.387856 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-e8770002-165c-4d3e-97bf-cc0d45898e94 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:33:58.387856 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:58.387856 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:58.387856 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:58.387856 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:58.387856 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:58.387856 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:58.387856 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:58.387856 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:58.387856 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:58.387856 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:58.387856 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:58.387856 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:33:58.387856 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:33:58.387856 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:33:58.387856 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:33:58.388897 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:33:58.388897 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:33:58.388897 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:33:58.388897 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:33:58.388897 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:58.388897 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:58.388897 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:58.388897 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:58.388897 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:58.388897 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:58.388897 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:58.388897 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:58.388897 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:58.388897 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:58.388897 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:58.388897 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:58.390025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:58.390025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:58.390025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:58.390025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:58.390025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:33:58.390025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:33:58.390025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:33:58.390025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:33:58.390025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:33:58.390025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:33:58.390025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:33:58.390025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:33:58.390025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 635/1276] 10.222.0.97 () {56 vars in 1096 bytes} [Tue Jul 26 14:33:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:33:58.395134 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacaadf-ca56-4dc5-8c24-620a3487d2c3 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:58.477028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:58.494703 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacaadf-ca56-4dc5-8c24-620a3487d2c3 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:58.528317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa69f80-4e0e-490d-bb18-ef41aae3b914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:58.528931 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa69f80-4e0e-490d-bb18-ef41aae3b914 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:58.529217 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa69f80-4e0e-490d-bb18-ef41aae3b914 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:58.529493 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa69f80-4e0e-490d-bb18-ef41aae3b914 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:58.530203 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa69f80-4e0e-490d-bb18-ef41aae3b914 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:58.530511 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa69f80-4e0e-490d-bb18-ef41aae3b914 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:58.531900 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa69f80-4e0e-490d-bb18-ef41aae3b914 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:58.540400 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 642/1277] 10.222.0.97 () {56 vars in 1099 bytes} [Tue Jul 26 14:33:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:58.551930 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8770002-165c-4d3e-97bf-cc0d45898e94 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:58.647229 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8770002-165c-4d3e-97bf-cc0d45898e94 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:58.680228 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36489cf-4707-4451-9dae-c0c19846f61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:58.680859 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36489cf-4707-4451-9dae-c0c19846f61b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:58.681139 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36489cf-4707-4451-9dae-c0c19846f61b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:58.681417 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36489cf-4707-4451-9dae-c0c19846f61b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:58.691742 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36489cf-4707-4451-9dae-c0c19846f61b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:58.693686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36489cf-4707-4451-9dae-c0c19846f61b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:58.708154 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-a36489cf-4707-4451-9dae-c0c19846f61b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:33:58.708154 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:58.708154 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:58.708154 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:58.708154 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:58.708154 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:58.708154 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:58.708154 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:58.708154 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:58.708154 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:58.708154 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:58.708154 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:58.708154 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:33:58.708154 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:33:58.708154 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:33:58.709260 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:33:58.709260 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:58.709260 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:58.709260 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:58.709260 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:58.709260 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:58.709260 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:58.709260 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:58.709260 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:58.709260 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:58.709260 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:58.709260 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:58.709260 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:58.709260 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:58.709260 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:58.709260 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:58.709260 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:58.715085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:33:58.715085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:33:58.715085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:33:58.715085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:33:58.715085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 636/1278] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:33:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:33:58.715499 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa69f80-4e0e-490d-bb18-ef41aae3b914 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:58.815900 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa69f80-4e0e-490d-bb18-ef41aae3b914 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:58.849516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d05c8-135f-4a14-81e9-d3ecd8d9612a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:58.850140 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d05c8-135f-4a14-81e9-d3ecd8d9612a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:58.850427 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d05c8-135f-4a14-81e9-d3ecd8d9612a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:58.850708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d05c8-135f-4a14-81e9-d3ecd8d9612a None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:58.851426 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2d05c8-135f-4a14-81e9-d3ecd8d9612a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:58.851740 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2d05c8-135f-4a14-81e9-d3ecd8d9612a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:58.853198 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2d05c8-135f-4a14-81e9-d3ecd8d9612a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:58.865169 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 643/1279] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:33:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:58.871143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36489cf-4707-4451-9dae-c0c19846f61b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:58.903861 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:58.970578 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36489cf-4707-4451-9dae-c0c19846f61b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:59.004374 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba379214-df5b-4893-9881-d804f498d7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:59.005004 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba379214-df5b-4893-9881-d804f498d7dd None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:59.005282 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba379214-df5b-4893-9881-d804f498d7dd None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:59.005553 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba379214-df5b-4893-9881-d804f498d7dd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:59.013342 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba379214-df5b-4893-9881-d804f498d7dd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:59.015379 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba379214-df5b-4893-9881-d804f498d7dd None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:59.026772 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-ba379214-df5b-4893-9881-d804f498d7dd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 14:33:59.026772 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:59.026772 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:59.026772 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:59.026772 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:59.026772 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:59.026772 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:59.026772 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:59.026772 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:59.026772 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:59.026772 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:59.026772 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:59.026772 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:33:59.026772 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:33:59.026772 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:33:59.026772 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:33:59.027989 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:59.027989 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:59.027989 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:33:59.027989 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:33:59.027989 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:33:59.027989 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:33:59.027989 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:59.027989 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:59.027989 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:59.027989 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:59.027989 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:59.027989 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:59.027989 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:59.027989 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:59.027989 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:59.027989 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:59.027989 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:59.027989 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:59.029320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:59.029320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:59.029320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:33:59.029320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:33:59.029320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:33:59.029320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:33:59.029320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:33:59.029320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:33:59.029320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 14:33:59.029320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:33:59.029320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 637/1280] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:33:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:33:59.033800 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d05c8-135f-4a14-81e9-d3ecd8d9612a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:59.038188 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:59.132951 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d05c8-135f-4a14-81e9-d3ecd8d9612a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:59.166817 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846d41c-c0d8-4eab-8455-cf0e507c04d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:59.167374 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846d41c-c0d8-4eab-8455-cf0e507c04d4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:59.167695 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846d41c-c0d8-4eab-8455-cf0e507c04d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:59.167974 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846d41c-c0d8-4eab-8455-cf0e507c04d4 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:59.168765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1846d41c-c0d8-4eab-8455-cf0e507c04d4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:59.169077 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1846d41c-c0d8-4eab-8455-cf0e507c04d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:59.170502 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1846d41c-c0d8-4eab-8455-cf0e507c04d4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:59.187500 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 644/1281] 10.222.0.97 () {56 vars in 1143 bytes} [Tue Jul 26 14:33:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:59.193969 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba379214-df5b-4893-9881-d804f498d7dd None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:59.282288 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:59.291326 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba379214-df5b-4893-9881-d804f498d7dd None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:59.326231 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5645d628-5138-4036-8cfb-501ca42f847d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:59.326858 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645d628-5138-4036-8cfb-501ca42f847d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:59.327298 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645d628-5138-4036-8cfb-501ca42f847d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:59.327611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645d628-5138-4036-8cfb-501ca42f847d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:59.397023 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5645d628-5138-4036-8cfb-501ca42f847d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:59.398476 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5645d628-5138-4036-8cfb-501ca42f847d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:59.486015 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:59.496872 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 638/1282] 10.222.0.97 () {58 vars in 1282 bytes} [Tue Jul 26 14:33:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:33:59.502761 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846d41c-c0d8-4eab-8455-cf0e507c04d4 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:59.565503 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:59.606105 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846d41c-c0d8-4eab-8455-cf0e507c04d4 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:59.641701 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c5437-a300-45c5-a8c9-042d89dfca4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:59.642345 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c5437-a300-45c5-a8c9-042d89dfca4a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:59.642624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c5437-a300-45c5-a8c9-042d89dfca4a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:59.642903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c5437-a300-45c5-a8c9-042d89dfca4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:59.652998 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c5437-a300-45c5-a8c9-042d89dfca4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:59.654442 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c5437-a300-45c5-a8c9-042d89dfca4a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:59.668610 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-9e0c5437-a300-45c5-a8c9-042d89dfca4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:33:59.668610 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:59.668610 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:59.668610 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:59.668610 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:59.668610 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:59.668610 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:59.668610 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:59.668610 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:59.668610 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:59.668610 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:59.668610 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:59.668610 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:33:59.668610 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:33:59.668610 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:33:59.669840 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:33:59.669840 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:59.669840 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:59.669840 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:59.669840 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:59.669840 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:59.669840 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:59.669840 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:59.669840 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:59.669840 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:59.669840 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:59.669840 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:59.669840 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:59.669840 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:59.669840 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:59.669840 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:59.669840 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:59.670743 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:33:59.670743 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:33:59.670743 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:33:59.670743 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:33:59.670743 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 645/1283] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:33:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:33:59.675711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5645d628-5138-4036-8cfb-501ca42f847d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:59.778724 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5645d628-5138-4036-8cfb-501ca42f847d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:59.812875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d68568c-fa43-4def-ad2e-0211b0ca0b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:59.813534 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68568c-fa43-4def-ad2e-0211b0ca0b3d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:59.813819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68568c-fa43-4def-ad2e-0211b0ca0b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:59.814096 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68568c-fa43-4def-ad2e-0211b0ca0b3d None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:59.814828 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d68568c-fa43-4def-ad2e-0211b0ca0b3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:33:59.815136 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d68568c-fa43-4def-ad2e-0211b0ca0b3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:59.816526 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d68568c-fa43-4def-ad2e-0211b0ca0b3d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:59.828398 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 639/1284] 10.222.0.97 () {56 vars in 1109 bytes} [Tue Jul 26 14:33:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:33:59.834116 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c5437-a300-45c5-a8c9-042d89dfca4a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:33:59.933698 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c5437-a300-45c5-a8c9-042d89dfca4a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:33:59.947495 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:33:59.967320 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ddb7b-43cd-4876-a02f-9ea9f42ef8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:33:59.967978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ddb7b-43cd-4876-a02f-9ea9f42ef8e6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:33:59.968263 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ddb7b-43cd-4876-a02f-9ea9f42ef8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:33:59.968538 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ddb7b-43cd-4876-a02f-9ea9f42ef8e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:33:59.975628 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ddb7b-43cd-4876-a02f-9ea9f42ef8e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:33:59.977050 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ddb7b-43cd-4876-a02f-9ea9f42ef8e6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:33:59.988034 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-783ddb7b-43cd-4876-a02f-9ea9f42ef8e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:33:59.988034 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:33:59.988034 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:33:59.988034 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:33:59.988034 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:33:59.988034 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:33:59.988034 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:33:59.988034 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:33:59.988034 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:33:59.988034 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:33:59.988034 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:33:59.988034 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:33:59.988034 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:33:59.988034 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:33:59.988034 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:33:59.989624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:33:59.989624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:33:59.989624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:33:59.989624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:33:59.989624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:33:59.989624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:33:59.989624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:33:59.989624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:33:59.989624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 26 14:33:59.989624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:33:59.989624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:33:59.989624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:33:59.989624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:33:59.989624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:33:59.989624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:33:59.989624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:33:59.989624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:33:59.994161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:33:59.994161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:33:59.994161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:33:59.994161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:33:59.994161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:33:59.994161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:33:59.994161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:33:59.994161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 26 14:33:59.994161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 646/1285] 10.222.0.97 () {56 vars in 1111 bytes} [Tue Jul 26 14:33:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:33:59.995733 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d68568c-fa43-4def-ad2e-0211b0ca0b3d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:34:00.045553 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:34:00.094343 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d68568c-fa43-4def-ad2e-0211b0ca0b3d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:34:00.127160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8312aa-d460-4793-99d0-9bfbb4197105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:34:00.127804 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8312aa-d460-4793-99d0-9bfbb4197105 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:34:00.128087 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8312aa-d460-4793-99d0-9bfbb4197105 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:34:00.128364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8312aa-d460-4793-99d0-9bfbb4197105 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:34:00.129915 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8312aa-d460-4793-99d0-9bfbb4197105 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:34:00.130220 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8312aa-d460-4793-99d0-9bfbb4197105 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:34:00.131664 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8312aa-d460-4793-99d0-9bfbb4197105 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:34:00.143265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 640/1286] 10.222.0.97 () {56 vars in 1148 bytes} [Tue Jul 26 14:33:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:34:00.150165 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ddb7b-43cd-4876-a02f-9ea9f42ef8e6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:34:00.248039 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ddb7b-43cd-4876-a02f-9ea9f42ef8e6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:34:00.280553 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc1024-f1de-4200-9100-caecb5b65708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:34:00.281182 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc1024-f1de-4200-9100-caecb5b65708 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:34:00.281448 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc1024-f1de-4200-9100-caecb5b65708 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:34:00.281726 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc1024-f1de-4200-9100-caecb5b65708 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:34:00.282504 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc1024-f1de-4200-9100-caecb5b65708 None admin] RBAC: Adding query filter params (role.id=51b2d10dfffb4b75b186f3d8b15e2a78, scope.project.id=c4c8da3011294783b2180ade1bfd9366, user.id=252c2d2939134416a3e8efb254f03c5b) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:34:00.282813 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc1024-f1de-4200-9100-caecb5b65708 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:34:00.284258 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc1024-f1de-4200-9100-caecb5b65708 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:34:00.290400 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 647/1287] 10.222.0.97 () {56 vars in 1374 bytes} [Tue Jul 26 14:34:00 2022] GET /identity/v3/role_assignments?user.id=252c2d2939134416a3e8efb254f03c5b&scope.project.id=c4c8da3011294783b2180ade1bfd9366&role.id=51b2d10dfffb4b75b186f3d8b15e2a78 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:34:02.309715 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ede7f9-782f-4f63-abc2-cb977c587b90 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:34:02.310231 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ede7f9-782f-4f63-abc2-cb977c587b90 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:34:02.310231 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ede7f9-782f-4f63-abc2-cb977c587b90 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:34:02.311604 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 641/1288] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:34:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:34:02.316808 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd787c4-56f2-422f-b6df-45bac7afa0df None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:34:02.317272 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd787c4-56f2-422f-b6df-45bac7afa0df None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:34:02.317482 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd787c4-56f2-422f-b6df-45bac7afa0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:34:02.336097 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-1fd787c4-56f2-422f-b6df-45bac7afa0df None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:34:02.337292 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1fd787c4-56f2-422f-b6df-45bac7afa0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:34:02.367378 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:34:02.406490 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 648/1289] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:34:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:34:02.428568 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ede7f9-782f-4f63-abc2-cb977c587b90 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:34:02.530927 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ede7f9-782f-4f63-abc2-cb977c587b90 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:34:02.541860 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:34:02.564740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8794e9-0d54-46e7-a4cb-d342f2af28e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:34:02.565319 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8794e9-0d54-46e7-a4cb-d342f2af28e4 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:34:02.565600 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8794e9-0d54-46e7-a4cb-d342f2af28e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:34:02.565874 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8794e9-0d54-46e7-a4cb-d342f2af28e4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:34:02.567017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8794e9-0d54-46e7-a4cb-d342f2af28e4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:34:02.568414 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8794e9-0d54-46e7-a4cb-d342f2af28e4 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:34:02.577293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-5d8794e9-0d54-46e7-a4cb-d342f2af28e4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 14:34:02.577293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:34:02.577293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:34:02.577293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:34:02.577293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:34:02.577293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:34:02.577293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:34:02.577293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:34:02.577293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 14:34:02.577293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:34:02.577293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:34:02.577293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:34:02.577293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:34:02.577293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:34:02.577293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:34:02.577293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:34:02.577293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:34:02.578625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:34:02.578625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:34:02.578625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:34:02.578625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:34:02.578625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:34:02.578625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:34:02.578625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:34:02.578625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:34:02.578625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:34:02.578625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:34:02.578625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:34:02.578625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:34:02.578625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:34:02.578625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:34:02.578625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:34:02.578625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:34:02.579879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:34:02.579879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:34:02.579879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:34:02.579879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:34:02.579879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:34:02.579879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:34:02.579879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:34:02.579879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:34:02.579879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:34:02.579879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:34:02.579879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:34:02.579879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:34:02.579879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 14:34:02.579879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:34:02.579879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 642/1290] 10.222.0.97 () {56 vars in 1117 bytes} [Tue Jul 26 14:34:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:34:02.585472 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd787c4-56f2-422f-b6df-45bac7afa0df None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:34:02.681176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd787c4-56f2-422f-b6df-45bac7afa0df None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:34:02.715393 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f46f6-9c96-4405-b368-bf081599c570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:34:02.716026 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f46f6-9c96-4405-b368-bf081599c570 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:34:02.716305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f46f6-9c96-4405-b368-bf081599c570 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:34:02.716575 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f46f6-9c96-4405-b368-bf081599c570 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:34:02.717306 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f46f6-9c96-4405-b368-bf081599c570 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:34:02.717611 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f46f6-9c96-4405-b368-bf081599c570 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:34:02.718959 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f46f6-9c96-4405-b368-bf081599c570 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:34:02.723813 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 649/1291] 10.222.0.97 () {56 vars in 1116 bytes} [Tue Jul 26 14:34:02 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:34:02.729527 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8794e9-0d54-46e7-a4cb-d342f2af28e4 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:34:02.832384 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8794e9-0d54-46e7-a4cb-d342f2af28e4 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:34:02.865911 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d72dc8-be8b-4685-9b78-27033eb654ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:34:02.866542 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d72dc8-be8b-4685-9b78-27033eb654ca None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:34:02.866810 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d72dc8-be8b-4685-9b78-27033eb654ca None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:34:02.867086 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d72dc8-be8b-4685-9b78-27033eb654ca None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:34:02.867802 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d72dc8-be8b-4685-9b78-27033eb654ca None admin] RBAC: Adding query filter params (type=placement) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:34:02.868110 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d72dc8-be8b-4685-9b78-27033eb654ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:34:02.869470 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d72dc8-be8b-4685-9b78-27033eb654ca None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:34:02.873854 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 643/1292] 10.222.0.97 () {56 vars in 1116 bytes} [Tue Jul 26 14:34:02 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:34:04.880807 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53caed7a-ad20-4cbd-b4b0-0ebd6fc6aaa2 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:34:04.881330 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53caed7a-ad20-4cbd-b4b0-0ebd6fc6aaa2 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:34:04.881449 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53caed7a-ad20-4cbd-b4b0-0ebd6fc6aaa2 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:34:04.882907 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 650/1293] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:34:04.888089 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393acc2-fa85-409f-9fe8-92290e8a8450 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:34:04.888517 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393acc2-fa85-409f-9fe8-92290e8a8450 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:34:04.888658 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393acc2-fa85-409f-9fe8-92290e8a8450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:34:04.906468 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a393acc2-fa85-409f-9fe8-92290e8a8450 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:34:04.907751 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a393acc2-fa85-409f-9fe8-92290e8a8450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:34:04.937792 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:34:04.973543 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 644/1294] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:34:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:34:04.996247 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53caed7a-ad20-4cbd-b4b0-0ebd6fc6aaa2 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:34:05.028324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:34:05.098978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53caed7a-ad20-4cbd-b4b0-0ebd6fc6aaa2 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:34:05.136173 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b072352-0f32-4d9e-9356-d72dd2c26650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:34:05.136173 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b072352-0f32-4d9e-9356-d72dd2c26650 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:34:05.136173 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b072352-0f32-4d9e-9356-d72dd2c26650 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:34:05.136173 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b072352-0f32-4d9e-9356-d72dd2c26650 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:34:05.136644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b072352-0f32-4d9e-9356-d72dd2c26650 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:34:05.136644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b072352-0f32-4d9e-9356-d72dd2c26650 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:34:05.218964 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 651/1295] 10.222.0.97 () {60 vars in 1141 bytes} [Tue Jul 26 14:34:04 2022] POST /identity/v3/services => generated 246 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:34:07.205218 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944aa03-e7b6-479d-a128-9fa6337c817f None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:34:07.205741 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944aa03-e7b6-479d-a128-9fa6337c817f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:34:07.205852 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944aa03-e7b6-479d-a128-9fa6337c817f None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:34:07.207248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 645/1296] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:34:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:34:07.212546 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639072d-05bc-456b-81eb-95ad4514a03b None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:34:07.212982 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639072d-05bc-456b-81eb-95ad4514a03b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:34:07.213079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639072d-05bc-456b-81eb-95ad4514a03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:34:07.231201 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-7639072d-05bc-456b-81eb-95ad4514a03b None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:34:07.232409 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7639072d-05bc-456b-81eb-95ad4514a03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:34:07.304761 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 652/1297] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:34:07 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:34:07.328650 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d944aa03-e7b6-479d-a128-9fa6337c817f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:34:07.429434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d944aa03-e7b6-479d-a128-9fa6337c817f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:34:07.464493 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523943c-ac05-4d70-9ae9-cef0ee6d0633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:34:07.465128 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523943c-ac05-4d70-9ae9-cef0ee6d0633 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:34:07.465411 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523943c-ac05-4d70-9ae9-cef0ee6d0633 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:34:07.465686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523943c-ac05-4d70-9ae9-cef0ee6d0633 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:34:07.466318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a523943c-ac05-4d70-9ae9-cef0ee6d0633 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:34:07.467801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a523943c-ac05-4d70-9ae9-cef0ee6d0633 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:34:07.477219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-a523943c-ac05-4d70-9ae9-cef0ee6d0633 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 14:34:07.477219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:34:07.477219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:34:07.477219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:34:07.477219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:34:07.477219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:34:07.477219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:34:07.477219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:34:07.477219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 14:34:07.477219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:34:07.477219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:34:07.477219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:34:07.477219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:34:07.477219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:34:07.477219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:34:07.477219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:34:07.477219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:34:07.478282 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:34:07.478282 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:34:07.478282 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:34:07.478282 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:34:07.478282 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:34:07.478282 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:34:07.478282 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:34:07.478282 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:34:07.478282 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:34:07.478282 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:34:07.478282 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:34:07.478282 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:34:07.478282 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:34:07.478282 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:34:07.478282 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:34:07.478282 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:34:07.479167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:34:07.479167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:34:07.479167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:34:07.479167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:34:07.479167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:34:07.479167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:34:07.479167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:34:07.479167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:34:07.479167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:34:07.479167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:34:07.479167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:34:07.479167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:34:07.479167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 14:34:07.479167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:34:07.479167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 646/1298] 10.222.0.97 () {56 vars in 1117 bytes} [Tue Jul 26 14:34:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:34:07.484903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7639072d-05bc-456b-81eb-95ad4514a03b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:34:07.503532 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:34:07.581313 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7639072d-05bc-456b-81eb-95ad4514a03b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:34:07.618439 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b25cb8c-818c-456b-a529-d6afafa73f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:34:07.619058 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25cb8c-818c-456b-a529-d6afafa73f36 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:34:07.619344 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25cb8c-818c-456b-a529-d6afafa73f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:34:07.619657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25cb8c-818c-456b-a529-d6afafa73f36 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:34:07.620329 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b25cb8c-818c-456b-a529-d6afafa73f36 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:34:07.620637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b25cb8c-818c-456b-a529-d6afafa73f36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:34:07.622002 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b25cb8c-818c-456b-a529-d6afafa73f36 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:34:07.627042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 653/1299] 10.222.0.97 () {56 vars in 1116 bytes} [Tue Jul 26 14:34:07 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:34:07.633994 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523943c-ac05-4d70-9ae9-cef0ee6d0633 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:34:07.665616 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:34:07.730647 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523943c-ac05-4d70-9ae9-cef0ee6d0633 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:34:07.766827 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed54c80-a51e-4635-a913-c65571f99d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:34:07.767486 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed54c80-a51e-4635-a913-c65571f99d4f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:34:07.767766 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed54c80-a51e-4635-a913-c65571f99d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:34:07.768037 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed54c80-a51e-4635-a913-c65571f99d4f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:34:07.768782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed54c80-a51e-4635-a913-c65571f99d4f None admin] RBAC: Adding query filter params (interface=public, service_id=6d0878084ef4496ea82bd14c648f8398, region_id=RegionOne) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:34:07.769708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed54c80-a51e-4635-a913-c65571f99d4f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:34:07.771069 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed54c80-a51e-4635-a913-c65571f99d4f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:34:07.776042 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 647/1300] 10.222.0.97 () {56 vars in 1251 bytes} [Tue Jul 26 14:34:07 2022] GET /identity/v3/endpoints?service_id=6d0878084ef4496ea82bd14c648f8398&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:34:07.781754 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b25cb8c-818c-456b-a529-d6afafa73f36 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:34:07.862095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:34:07.880381 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b25cb8c-818c-456b-a529-d6afafa73f36 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:34:07.913035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89552136-2d90-44fe-8846-0a6d4f2484e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:34:07.913639 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89552136-2d90-44fe-8846-0a6d4f2484e2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:34:07.913920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89552136-2d90-44fe-8846-0a6d4f2484e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:34:07.914193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89552136-2d90-44fe-8846-0a6d4f2484e2 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:34:07.914883 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89552136-2d90-44fe-8846-0a6d4f2484e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:34:07.916318 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89552136-2d90-44fe-8846-0a6d4f2484e2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:34:07.923080 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 654/1301] 10.222.0.97 () {56 vars in 1087 bytes} [Tue Jul 26 14:34:07 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:34:09.942888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc6197-2c08-41f2-b2a1-bf15dcec9b5a None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:34:10.023478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc6197-2c08-41f2-b2a1-bf15dcec9b5a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:34:10.023478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc6197-2c08-41f2-b2a1-bf15dcec9b5a None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:34:10.023478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 648/1302] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:34:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:34:10.023789 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59332a48-863e-4879-acfc-dc1aef3ad44e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:34:10.023789 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59332a48-863e-4879-acfc-dc1aef3ad44e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:34:10.023789 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59332a48-863e-4879-acfc-dc1aef3ad44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:34:10.023789 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-59332a48-863e-4879-acfc-dc1aef3ad44e None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:34:10.023789 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-59332a48-863e-4879-acfc-dc1aef3ad44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:34:10.042840 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 655/1303] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:34:09 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:34:10.065011 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc6197-2c08-41f2-b2a1-bf15dcec9b5a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:34:10.159473 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:34:10.166063 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc6197-2c08-41f2-b2a1-bf15dcec9b5a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:34:10.199025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505bb42-acaf-4a5b-995e-9af54c5a55d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:34:10.199637 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505bb42-acaf-4a5b-995e-9af54c5a55d8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:34:10.199919 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505bb42-acaf-4a5b-995e-9af54c5a55d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:34:10.200194 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505bb42-acaf-4a5b-995e-9af54c5a55d8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:34:10.200806 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505bb42-acaf-4a5b-995e-9af54c5a55d8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:34:10.202148 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505bb42-acaf-4a5b-995e-9af54c5a55d8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:34:10.211362 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-5505bb42-acaf-4a5b-995e-9af54c5a55d8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 14:34:10.211362 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:34:10.211362 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:34:10.211362 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:34:10.211362 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:34:10.211362 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:34:10.211362 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:34:10.211362 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:34:10.211362 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 14:34:10.211362 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:34:10.211362 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:34:10.211362 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:34:10.211362 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:34:10.211362 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:34:10.211362 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:34:10.211362 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:34:10.211362 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:34:10.212481 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:34:10.212481 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:34:10.212481 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:34:10.212481 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:34:10.212481 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:34:10.212481 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:34:10.212481 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:34:10.212481 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:34:10.212481 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:34:10.212481 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:34:10.212481 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:34:10.212481 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:34:10.212481 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:34:10.212481 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:34:10.212481 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:34:10.212481 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:34:10.213740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:34:10.213740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 26 14:34:10.213740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:34:10.213740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:34:10.213740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:34:10.213740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:34:10.213740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:34:10.213740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:34:10.213740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:34:10.213740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:34:10.213740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:34:10.213740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:34:10.213740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 14:34:10.213740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 26 14:34:10.213740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 649/1304] 10.222.0.97 () {56 vars in 1117 bytes} [Tue Jul 26 14:34:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:34:10.218581 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59332a48-863e-4879-acfc-dc1aef3ad44e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:34:10.293207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:34:10.314543 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59332a48-863e-4879-acfc-dc1aef3ad44e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:34:10.347923 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c588792c-2982-444e-87f8-f86b3a139ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:34:10.348542 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c588792c-2982-444e-87f8-f86b3a139ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:34:10.348882 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c588792c-2982-444e-87f8-f86b3a139ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:34:10.349103 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c588792c-2982-444e-87f8-f86b3a139ed7 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:34:10.349776 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c588792c-2982-444e-87f8-f86b3a139ed7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:34:10.350082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c588792c-2982-444e-87f8-f86b3a139ed7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:34:10.351456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c588792c-2982-444e-87f8-f86b3a139ed7 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:34:10.357256 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 656/1305] 10.222.0.97 () {56 vars in 1116 bytes} [Tue Jul 26 14:34:10 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:34:10.362967 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505bb42-acaf-4a5b-995e-9af54c5a55d8 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:34:10.462204 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505bb42-acaf-4a5b-995e-9af54c5a55d8 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:34:10.495282 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87867f-b1b3-496c-bd98-361fa5e3bd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:34:10.495938 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87867f-b1b3-496c-bd98-361fa5e3bd87 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:34:10.496222 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87867f-b1b3-496c-bd98-361fa5e3bd87 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:34:10.496502 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87867f-b1b3-496c-bd98-361fa5e3bd87 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:34:10.497210 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87867f-b1b3-496c-bd98-361fa5e3bd87 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:34:10.498574 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87867f-b1b3-496c-bd98-361fa5e3bd87 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:34:10.617735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 650/1306] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:34:10 2022] POST /identity/v3/endpoints => generated 331 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:34:20.357716 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=30 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:34:20.613313 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:55.100699 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680a43b-f351-40b1-aeea-a2be90df9579 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:55.100699 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680a43b-f351-40b1-aeea-a2be90df9579 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:55.100699 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680a43b-f351-40b1-aeea-a2be90df9579 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:55.100699 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 657/1307] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:42:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:42:55.111936 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c6cba-38c9-4dd9-b95e-1087c417cc28 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:55.111936 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c6cba-38c9-4dd9-b95e-1087c417cc28 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:55.111936 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c6cba-38c9-4dd9-b95e-1087c417cc28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:55.278692 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-903c6cba-38c9-4dd9-b95e-1087c417cc28 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:55.279974 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-903c6cba-38c9-4dd9-b95e-1087c417cc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:55.399090 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 651/1308] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:42:55.453093 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43233d-cad2-4ab6-a6a1-48fd701da455 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:55.453346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43233d-cad2-4ab6-a6a1-48fd701da455 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:55.453810 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43233d-cad2-4ab6-a6a1-48fd701da455 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:55.455177 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 658/1309] 10.222.0.97 () {54 vars in 965 bytes} [Tue Jul 26 14:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:42:55.461477 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5c7dd-9ba5-449d-910c-4a7addfce803 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:55.461984 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5c7dd-9ba5-449d-910c-4a7addfce803 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:55.462365 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5c7dd-9ba5-449d-910c-4a7addfce803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:55.546128 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-68f5c7dd-9ba5-449d-910c-4a7addfce803 None None] MFA Rules not processed for user `c81180fb4ff04c1186a21e43fb597a91`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:55.547453 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-68f5c7dd-9ba5-449d-910c-4a7addfce803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:55.569922 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:55.631310 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 652/1310] 10.222.0.97 () {58 vars in 1062 bytes} [Tue Jul 26 14:42:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:42:55.641356 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43233d-cad2-4ab6-a6a1-48fd701da455 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:55.643234 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2b43233d-cad2-4ab6-a6a1-48fd701da455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:55.779113 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43233d-cad2-4ab6-a6a1-48fd701da455 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:55.873191 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe1a20-4d77-468f-ad83-e345cb8b3006 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:55.873932 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe1a20-4d77-468f-ad83-e345cb8b3006 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:55.874308 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe1a20-4d77-468f-ad83-e345cb8b3006 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:55.874659 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe1a20-4d77-468f-ad83-e345cb8b3006 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:55.876848 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-18fe1a20-4d77-468f-ad83-e345cb8b3006 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:55.945248 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:56.006168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe1a20-4d77-468f-ad83-e345cb8b3006 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:56.007904 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe1a20-4d77-468f-ad83-e345cb8b3006 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:56.028685 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 659/1311] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:42:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:42:58.365992 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021d0e9-ab44-42e5-ad84-8cb22e04ac9e None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:58.366498 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021d0e9-ab44-42e5-ad84-8cb22e04ac9e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:58.366498 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021d0e9-ab44-42e5-ad84-8cb22e04ac9e None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:58.368132 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 653/1312] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:42:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:42:58.373612 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3ef78-979f-4468-9729-754d255d0a94 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:58.374311 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3ef78-979f-4468-9729-754d255d0a94 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:58.374311 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3ef78-979f-4468-9729-754d255d0a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:58.460580 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-bfd3ef78-979f-4468-9729-754d255d0a94 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:58.461798 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-bfd3ef78-979f-4468-9729-754d255d0a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:58.577750 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 660/1313] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:42:58.619070 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021d0e9-ab44-42e5-ad84-8cb22e04ac9e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:58.620645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c021d0e9-ab44-42e5-ad84-8cb22e04ac9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:58.730456 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021d0e9-ab44-42e5-ad84-8cb22e04ac9e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:58.802139 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:58.804682 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d17398-8967-42e8-8c6d-242fa32067e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:58.805304 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d17398-8967-42e8-8c6d-242fa32067e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:58.805583 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d17398-8967-42e8-8c6d-242fa32067e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:58.805855 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d17398-8967-42e8-8c6d-242fa32067e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:58.807523 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-07d17398-8967-42e8-8c6d-242fa32067e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:58.918077 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d17398-8967-42e8-8c6d-242fa32067e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:58.919459 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d17398-8967-42e8-8c6d-242fa32067e6 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:58.938905 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 654/1314] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:42:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:43:01.562475 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2a9fe-6f83-4bb4-bdf6-b733e06b6347 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:01.563019 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2a9fe-6f83-4bb4-bdf6-b733e06b6347 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:01.563019 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2a9fe-6f83-4bb4-bdf6-b733e06b6347 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:01.564434 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 661/1315] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:43:01.570653 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea43e80-3754-4090-8409-18d893a7ba55 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:01.571056 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea43e80-3754-4090-8409-18d893a7ba55 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:01.571208 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea43e80-3754-4090-8409-18d893a7ba55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:01.637690 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-dea43e80-3754-4090-8409-18d893a7ba55 None None] MFA Rules not processed for user `45da4bf9f6e64bc9bfa6f4e5beafb859`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:43:01.638899 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-dea43e80-3754-4090-8409-18d893a7ba55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:01.690965 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:01.735028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 655/1316] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:43:01 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:43:01.787436 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc190c-5a6f-42dc-a2d2-364576ca8e27 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:01.787723 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc190c-5a6f-42dc-a2d2-364576ca8e27 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:01.788003 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc190c-5a6f-42dc-a2d2-364576ca8e27 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:01.789360 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 662/1317] 10.222.0.97 () {54 vars in 965 bytes} [Tue Jul 26 14:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:43:01.795025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fac348-39d3-426c-b440-76440fa4da51 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:01.795309 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fac348-39d3-426c-b440-76440fa4da51 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:01.795614 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fac348-39d3-426c-b440-76440fa4da51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:01.873926 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-92fac348-39d3-426c-b440-76440fa4da51 None None] MFA Rules not processed for user `c81180fb4ff04c1186a21e43fb597a91`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:43:01.875057 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-92fac348-39d3-426c-b440-76440fa4da51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:01.951235 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 656/1318] 10.222.0.97 () {58 vars in 1062 bytes} [Tue Jul 26 14:43:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:43:01.960839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc190c-5a6f-42dc-a2d2-364576ca8e27 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:01.962351 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-31dc190c-5a6f-42dc-a2d2-364576ca8e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:01.972143 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:02.066322 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc190c-5a6f-42dc-a2d2-364576ca8e27 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:02.139214 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a1bef-437e-4b41-8158-200be2fd154b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:02.139786 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a1bef-437e-4b41-8158-200be2fd154b service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:02.140074 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a1bef-437e-4b41-8158-200be2fd154b service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:02.140351 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a1bef-437e-4b41-8158-200be2fd154b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:02.141979 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a95a1bef-437e-4b41-8158-200be2fd154b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:02.204640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:02.251730 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a1bef-437e-4b41-8158-200be2fd154b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:02.253161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a1bef-437e-4b41-8158-200be2fd154b service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:02.271844 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 663/1319] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:43:01 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:43:05.623683 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123689ae-fb79-4bea-a2cf-e8772b848fbd None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:05.624290 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123689ae-fb79-4bea-a2cf-e8772b848fbd None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:05.624290 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123689ae-fb79-4bea-a2cf-e8772b848fbd None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:05.625645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 657/1320] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:43:05.631561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a162168-9599-4ce9-b78f-94a15638f93b None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:05.632011 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a162168-9599-4ce9-b78f-94a15638f93b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:05.632124 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a162168-9599-4ce9-b78f-94a15638f93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:05.697445 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3a162168-9599-4ce9-b78f-94a15638f93b None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:43:05.698688 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3a162168-9599-4ce9-b78f-94a15638f93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:05.795390 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 664/1321] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:43:05 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:43:05.837335 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123689ae-fb79-4bea-a2cf-e8772b848fbd None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:05.838867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-123689ae-fb79-4bea-a2cf-e8772b848fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:05.890330 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:05.942377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123689ae-fb79-4bea-a2cf-e8772b848fbd None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:06.016114 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed2281-5542-41e3-a574-24aae587b4cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:06.016733 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed2281-5542-41e3-a574-24aae587b4cf service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:06.017017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed2281-5542-41e3-a574-24aae587b4cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:06.017296 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed2281-5542-41e3-a574-24aae587b4cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:06.018921 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d9ed2281-5542-41e3-a574-24aae587b4cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:06.115339 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:06.128476 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed2281-5542-41e3-a574-24aae587b4cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:06.130521 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed2281-5542-41e3-a574-24aae587b4cf service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:06.148581 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 658/1322] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:43:05 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:43:09.557175 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda2657-94f2-4ee9-9755-5e635488109c None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:09.557704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda2657-94f2-4ee9-9755-5e635488109c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:09.557819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda2657-94f2-4ee9-9755-5e635488109c None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:09.559142 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 665/1323] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:43:09.564532 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae09b3-d2dd-4475-aba0-aa19d59b3565 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:09.564934 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae09b3-d2dd-4475-aba0-aa19d59b3565 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:09.565094 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae09b3-d2dd-4475-aba0-aa19d59b3565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:09.632787 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e6ae09b3-d2dd-4475-aba0-aa19d59b3565 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:43:09.634328 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e6ae09b3-d2dd-4475-aba0-aa19d59b3565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:09.735667 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 659/1324] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:43:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:43:09.769204 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda2657-94f2-4ee9-9755-5e635488109c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:09.770833 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5fda2657-94f2-4ee9-9755-5e635488109c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:09.790253 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:09.886147 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda2657-94f2-4ee9-9755-5e635488109c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:09.965509 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400833b9-163c-4d19-9111-6a22261ad9a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:09.966357 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400833b9-163c-4d19-9111-6a22261ad9a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:09.966645 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400833b9-163c-4d19-9111-6a22261ad9a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:09.967013 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400833b9-163c-4d19-9111-6a22261ad9a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:09.968600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-400833b9-163c-4d19-9111-6a22261ad9a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:10.044097 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:10.083143 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400833b9-163c-4d19-9111-6a22261ad9a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:10.084621 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400833b9-163c-4d19-9111-6a22261ad9a2 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:10.102824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 666/1325] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:43:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:43:12.162361 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2f60a-897c-4a39-91cb-81e83b60bbc5 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:12.162882 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2f60a-897c-4a39-91cb-81e83b60bbc5 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:12.162988 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2f60a-897c-4a39-91cb-81e83b60bbc5 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:12.164517 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 660/1326] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:43:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:43:12.169923 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292785c1-ec1b-4408-9c56-7b6aa8a1237e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:12.170338 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292785c1-ec1b-4408-9c56-7b6aa8a1237e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:12.170659 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292785c1-ec1b-4408-9c56-7b6aa8a1237e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:12.239414 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-292785c1-ec1b-4408-9c56-7b6aa8a1237e None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:43:12.240631 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-292785c1-ec1b-4408-9c56-7b6aa8a1237e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:12.339353 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 667/1327] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:43:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:43:12.376689 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b2f60a-897c-4a39-91cb-81e83b60bbc5 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:12.378301 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-69b2f60a-897c-4a39-91cb-81e83b60bbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:12.440640 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:12.483485 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b2f60a-897c-4a39-91cb-81e83b60bbc5 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:12.560338 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aaeb5a-62df-4b08-a423-759811c9f66e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:12.560967 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aaeb5a-62df-4b08-a423-759811c9f66e service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:12.561247 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aaeb5a-62df-4b08-a423-759811c9f66e service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:12.561522 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aaeb5a-62df-4b08-a423-759811c9f66e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:12.563148 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e2aaeb5a-62df-4b08-a423-759811c9f66e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:12.676718 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:12.679686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aaeb5a-62df-4b08-a423-759811c9f66e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:12.681199 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aaeb5a-62df-4b08-a423-759811c9f66e service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:12.702834 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 661/1328] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:43:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:43:14.773198 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38887d70-9e7f-418b-ba30-0927733185bd None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:14.773684 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38887d70-9e7f-418b-ba30-0927733185bd None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:14.773922 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38887d70-9e7f-418b-ba30-0927733185bd None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:14.775453 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 668/1329] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:43:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:43:14.781101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc9642-23d1-4a5d-984f-f8c6479f59dc None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:14.781566 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc9642-23d1-4a5d-984f-f8c6479f59dc None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:14.781663 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc9642-23d1-4a5d-984f-f8c6479f59dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:14.853008 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-afcc9642-23d1-4a5d-984f-f8c6479f59dc None None] MFA Rules not processed for user `45da4bf9f6e64bc9bfa6f4e5beafb859`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:43:14.860362 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-afcc9642-23d1-4a5d-984f-f8c6479f59dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:14.952115 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 662/1330] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:43:14 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:43:14.990535 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38887d70-9e7f-418b-ba30-0927733185bd None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:14.992114 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-38887d70-9e7f-418b-ba30-0927733185bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:15.024827 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:15.100980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38887d70-9e7f-418b-ba30-0927733185bd None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:15.180275 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a75713-76ec-48e1-a97c-a9835e961157 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:15.180887 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a75713-76ec-48e1-a97c-a9835e961157 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:15.181168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a75713-76ec-48e1-a97c-a9835e961157 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:15.181441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a75713-76ec-48e1-a97c-a9835e961157 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:15.183040 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b9a75713-76ec-48e1-a97c-a9835e961157 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:15.261647 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:15.294258 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a75713-76ec-48e1-a97c-a9835e961157 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:15.295744 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a75713-76ec-48e1-a97c-a9835e961157 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:15.314528 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 669/1331] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:43:14 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:43:17.873930 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28a420-9996-485f-8560-3d359ae4daf4 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:17.874499 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28a420-9996-485f-8560-3d359ae4daf4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:17.874499 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28a420-9996-485f-8560-3d359ae4daf4 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:17.875907 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 663/1332] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:43:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:43:17.882358 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5382d-398e-452c-88a1-962756ab609c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:17.883091 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5382d-398e-452c-88a1-962756ab609c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:17.883258 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5382d-398e-452c-88a1-962756ab609c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:17.950744 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-aac5382d-398e-452c-88a1-962756ab609c None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:43:17.952055 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-aac5382d-398e-452c-88a1-962756ab609c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:18.055461 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 670/1333] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:43:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:43:18.090898 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28a420-9996-485f-8560-3d359ae4daf4 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:18.092486 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4f28a420-9996-485f-8560-3d359ae4daf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:18.162956 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:18.196667 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28a420-9996-485f-8560-3d359ae4daf4 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:18.271159 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc264c43-aa1b-44bc-b281-06cfdccc1a70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:18.271849 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc264c43-aa1b-44bc-b281-06cfdccc1a70 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:18.272144 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc264c43-aa1b-44bc-b281-06cfdccc1a70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:18.272421 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc264c43-aa1b-44bc-b281-06cfdccc1a70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:18.274052 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-dc264c43-aa1b-44bc-b281-06cfdccc1a70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:18.383368 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc264c43-aa1b-44bc-b281-06cfdccc1a70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:18.384856 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc264c43-aa1b-44bc-b281-06cfdccc1a70 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:18.396189 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:18.407303 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 664/1334] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:43:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:43:20.467870 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f6e53-6d00-4299-a932-1c2452c0f31e None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:20.468374 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f6e53-6d00-4299-a932-1c2452c0f31e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:20.468374 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f6e53-6d00-4299-a932-1c2452c0f31e None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:20.469681 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 671/1335] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:43:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:43:20.475425 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be1e97-524f-4465-9edf-48b24a5e819d None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:20.475871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be1e97-524f-4465-9edf-48b24a5e819d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:20.476003 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be1e97-524f-4465-9edf-48b24a5e819d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:20.543421 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a8be1e97-524f-4465-9edf-48b24a5e819d None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:43:20.544638 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a8be1e97-524f-4465-9edf-48b24a5e819d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:20.644962 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 665/1336] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:43:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:43:20.685959 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f6e53-6d00-4299-a932-1c2452c0f31e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:20.687536 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-581f6e53-6d00-4299-a932-1c2452c0f31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:20.727117 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:20.797801 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f6e53-6d00-4299-a932-1c2452c0f31e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:20.870479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d80c7-114f-4887-ad44-2b1f67fa77fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:20.871088 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d80c7-114f-4887-ad44-2b1f67fa77fd service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:20.871373 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d80c7-114f-4887-ad44-2b1f67fa77fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:20.871693 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d80c7-114f-4887-ad44-2b1f67fa77fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:20.873312 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9f2d80c7-114f-4887-ad44-2b1f67fa77fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:20.957128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:20.983536 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d80c7-114f-4887-ad44-2b1f67fa77fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:20.984920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d80c7-114f-4887-ad44-2b1f67fa77fd service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:21.008416 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 672/1337] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:43:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:43:23.887901 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8415b29-e5fd-4a80-919e-aff9f07f32c9 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:23.888464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8415b29-e5fd-4a80-919e-aff9f07f32c9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:23.888572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8415b29-e5fd-4a80-919e-aff9f07f32c9 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:23.889957 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 666/1338] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:43:23.895715 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02585680-ae24-4f13-bafa-c0b13c247dd3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:23.896129 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02585680-ae24-4f13-bafa-c0b13c247dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:23.896230 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02585680-ae24-4f13-bafa-c0b13c247dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:23.961584 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-02585680-ae24-4f13-bafa-c0b13c247dd3 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:43:23.962792 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-02585680-ae24-4f13-bafa-c0b13c247dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:24.059080 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 673/1339] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:43:23 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:43:24.094989 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8415b29-e5fd-4a80-919e-aff9f07f32c9 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:24.097180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a8415b29-e5fd-4a80-919e-aff9f07f32c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:24.177836 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:24.205367 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8415b29-e5fd-4a80-919e-aff9f07f32c9 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:24.287845 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16031f71-c48d-4aaa-808e-263ae3ddec39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:24.288464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16031f71-c48d-4aaa-808e-263ae3ddec39 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:24.288741 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16031f71-c48d-4aaa-808e-263ae3ddec39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:24.289011 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16031f71-c48d-4aaa-808e-263ae3ddec39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:24.290663 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-16031f71-c48d-4aaa-808e-263ae3ddec39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:24.408176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16031f71-c48d-4aaa-808e-263ae3ddec39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:24.410166 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16031f71-c48d-4aaa-808e-263ae3ddec39 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:24.429138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 667/1340] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:43:24 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:43:28.912999 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a17ff-e9ce-44fb-b3ac-0443c264c815 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:28.913538 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a17ff-e9ce-44fb-b3ac-0443c264c815 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:28.913538 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a17ff-e9ce-44fb-b3ac-0443c264c815 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:28.914957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 674/1341] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:43:28.920579 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c189b-4f85-4356-a645-dd61d5b98005 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:28.921525 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c189b-4f85-4356-a645-dd61d5b98005 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:28.921751 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c189b-4f85-4356-a645-dd61d5b98005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:28.927587 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:29.017189 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-0d4c189b-4f85-4356-a645-dd61d5b98005 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:43:29.018679 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0d4c189b-4f85-4356-a645-dd61d5b98005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:29.136434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 668/1342] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:43:29.176697 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a17ff-e9ce-44fb-b3ac-0443c264c815 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:29.178338 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-816a17ff-e9ce-44fb-b3ac-0443c264c815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:29.229474 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:29.289749 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a17ff-e9ce-44fb-b3ac-0443c264c815 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:29.371805 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e2bae-38bb-490f-b9eb-b5e5f06e6001 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:29.372479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e2bae-38bb-490f-b9eb-b5e5f06e6001 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:29.372765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e2bae-38bb-490f-b9eb-b5e5f06e6001 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:29.373046 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e2bae-38bb-490f-b9eb-b5e5f06e6001 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:29.374675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-252e2bae-38bb-490f-b9eb-b5e5f06e6001 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:29.472553 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:29.489162 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252e2bae-38bb-490f-b9eb-b5e5f06e6001 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:29.490612 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252e2bae-38bb-490f-b9eb-b5e5f06e6001 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:29.509444 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 675/1343] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:43:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:43:32.778691 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ccd06-3c7f-462a-a050-839cbf18234e None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:32.779230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ccd06-3c7f-462a-a050-839cbf18234e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:32.779339 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ccd06-3c7f-462a-a050-839cbf18234e None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:32.780819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 669/1344] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:43:32.786816 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e46976-eac9-42d0-9a88-5d1ea74dde1c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:32.787243 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e46976-eac9-42d0-9a88-5d1ea74dde1c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:32.787367 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e46976-eac9-42d0-9a88-5d1ea74dde1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:32.858001 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-42e46976-eac9-42d0-9a88-5d1ea74dde1c None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:43:32.859264 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-42e46976-eac9-42d0-9a88-5d1ea74dde1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:32.958003 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 676/1345] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:43:32 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:43:32.996489 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ccd06-3c7f-462a-a050-839cbf18234e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:32.998124 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-023ccd06-3c7f-462a-a050-839cbf18234e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:33.097105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:33.114090 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ccd06-3c7f-462a-a050-839cbf18234e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:33.194027 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df58bc56-bec2-4d5c-b45f-3cce63bc8b9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:33.194660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df58bc56-bec2-4d5c-b45f-3cce63bc8b9d service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:33.194943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df58bc56-bec2-4d5c-b45f-3cce63bc8b9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:33.195221 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df58bc56-bec2-4d5c-b45f-3cce63bc8b9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:33.196904 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-df58bc56-bec2-4d5c-b45f-3cce63bc8b9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:33.311462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df58bc56-bec2-4d5c-b45f-3cce63bc8b9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:33.312822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df58bc56-bec2-4d5c-b45f-3cce63bc8b9d service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:33.333720 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 670/1346] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:43:32 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:43:38.218262 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2774d6-1302-4952-83d2-ccca6f48e6ca None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:38.218799 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2774d6-1302-4952-83d2-ccca6f48e6ca None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:38.218799 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2774d6-1302-4952-83d2-ccca6f48e6ca None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:38.220222 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 677/1347] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:43:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:43:38.225879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb367f38-29c8-47cb-8ccb-b2795b373d63 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:38.226393 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb367f38-29c8-47cb-8ccb-b2795b373d63 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:38.226640 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb367f38-29c8-47cb-8ccb-b2795b373d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:38.244272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:38.300978 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-fb367f38-29c8-47cb-8ccb-b2795b373d63 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:43:38.302244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-fb367f38-29c8-47cb-8ccb-b2795b373d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:38.409236 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 671/1348] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:43:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:43:38.447522 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2774d6-1302-4952-83d2-ccca6f48e6ca None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:38.449126 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2e2774d6-1302-4952-83d2-ccca6f48e6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:38.509704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:38.558486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2774d6-1302-4952-83d2-ccca6f48e6ca None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:38.640745 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592b9e7-268c-4c49-bbee-f4dca2217714 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:38.641381 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592b9e7-268c-4c49-bbee-f4dca2217714 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:38.641667 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592b9e7-268c-4c49-bbee-f4dca2217714 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:38.641943 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592b9e7-268c-4c49-bbee-f4dca2217714 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:38.644375 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7592b9e7-268c-4c49-bbee-f4dca2217714 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:38.757549 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:38.761183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7592b9e7-268c-4c49-bbee-f4dca2217714 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:38.765495 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7592b9e7-268c-4c49-bbee-f4dca2217714 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:38.783140 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 678/1349] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:43:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:43:43.709034 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b565911-e79d-4dbf-ae3c-5086fc92b8dd None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:43.709951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b565911-e79d-4dbf-ae3c-5086fc92b8dd None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:43.710461 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b565911-e79d-4dbf-ae3c-5086fc92b8dd None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:43.712501 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 672/1350] 10.222.0.97 () {54 vars in 921 bytes} [Tue Jul 26 14:43:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:43:43.721676 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9f366-0571-4a40-859e-725654db845b None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:43.721676 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9f366-0571-4a40-859e-725654db845b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:43.722168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9f366-0571-4a40-859e-725654db845b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:43.818008 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d3c9f366-0571-4a40-859e-725654db845b None None] MFA Rules not processed for user `252c2d2939134416a3e8efb254f03c5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:43:43.819636 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d3c9f366-0571-4a40-859e-725654db845b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:43.969469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 679/1351] 10.222.0.97 () {58 vars in 1018 bytes} [Tue Jul 26 14:43:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:43:48.156752 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ea454-7adc-4a09-8ae2-c7ba54215840 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:48.157843 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ea454-7adc-4a09-8ae2-c7ba54215840 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:48.157843 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ea454-7adc-4a09-8ae2-c7ba54215840 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:48.159447 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 673/1352] 10.222.0.97 () {54 vars in 921 bytes} [Tue Jul 26 14:43:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:43:48.164581 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7273b3-746f-494c-a04c-8395c4cfbfae None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:48.165046 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7273b3-746f-494c-a04c-8395c4cfbfae None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:48.165160 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7273b3-746f-494c-a04c-8395c4cfbfae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:48.228401 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:48.252200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-de7273b3-746f-494c-a04c-8395c4cfbfae None None] MFA Rules not processed for user `252c2d2939134416a3e8efb254f03c5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:43:48.253404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-de7273b3-746f-494c-a04c-8395c4cfbfae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:48.365337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 680/1353] 10.222.0.97 () {58 vars in 1018 bytes} [Tue Jul 26 14:43:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:43:48.406249 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=59 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:49.733944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e146d8-434e-46a1-a4d5-2c27a8859194 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:49.734607 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e146d8-434e-46a1-a4d5-2c27a8859194 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:49.734607 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e146d8-434e-46a1-a4d5-2c27a8859194 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:49.735937 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 674/1354] 10.222.0.97 () {54 vars in 921 bytes} [Tue Jul 26 14:43:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:43:49.741159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998e762-b87d-44d2-bc2e-6143b4093389 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:49.741670 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998e762-b87d-44d2-bc2e-6143b4093389 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:49.741779 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998e762-b87d-44d2-bc2e-6143b4093389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:49.821683 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-9998e762-b87d-44d2-bc2e-6143b4093389 None None] MFA Rules not processed for user `252c2d2939134416a3e8efb254f03c5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:43:49.822898 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9998e762-b87d-44d2-bc2e-6143b4093389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:49.864874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:49.933268 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 681/1355] 10.222.0.97 () {58 vars in 1018 bytes} [Tue Jul 26 14:43:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:43:53.421459 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788ff15-73a1-4823-a7fc-c76846041d41 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:53.421980 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788ff15-73a1-4823-a7fc-c76846041d41 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:53.421980 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788ff15-73a1-4823-a7fc-c76846041d41 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:53.425259 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 675/1356] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:43:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:43:53.431638 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f780655-34e3-4222-89e2-7b9df231a6c7 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:53.432901 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f780655-34e3-4222-89e2-7b9df231a6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:53.433543 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f780655-34e3-4222-89e2-7b9df231a6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:53.513446 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6f780655-34e3-4222-89e2-7b9df231a6c7 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:43:53.515060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6f780655-34e3-4222-89e2-7b9df231a6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:53.647470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 682/1357] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:43:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:43:53.690562 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23968a0a-5170-4f0c-bbe0-c073cacab856 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:53.691620 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23968a0a-5170-4f0c-bbe0-c073cacab856 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:53.692205 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23968a0a-5170-4f0c-bbe0-c073cacab856 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:53.693995 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 676/1358] 10.222.0.97 () {54 vars in 962 bytes} [Tue Jul 26 14:43:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:43:53.706907 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ecd841-ff20-4659-b550-4d84e93850cd None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:53.707295 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ecd841-ff20-4659-b550-4d84e93850cd None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:53.712076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ecd841-ff20-4659-b550-4d84e93850cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:53.725068 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:53.795492 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-20ecd841-ff20-4659-b550-4d84e93850cd None None] MFA Rules not processed for user `266abef6cda5495b897170f92043aa03`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:43:53.796710 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-20ecd841-ff20-4659-b550-4d84e93850cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:53.929669 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 683/1359] 10.222.0.97 () {58 vars in 1059 bytes} [Tue Jul 26 14:43:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:43:53.940166 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23968a0a-5170-4f0c-bbe0-c073cacab856 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:53.942823 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-23968a0a-5170-4f0c-bbe0-c073cacab856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:54.127570 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23968a0a-5170-4f0c-bbe0-c073cacab856 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:54.226091 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:54.276809 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35010710-777d-4732-a858-846e36cb46e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:54.277469 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35010710-777d-4732-a858-846e36cb46e1 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:54.277755 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35010710-777d-4732-a858-846e36cb46e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:54.278033 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35010710-777d-4732-a858-846e36cb46e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:54.279845 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-35010710-777d-4732-a858-846e36cb46e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:54.384391 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35010710-777d-4732-a858-846e36cb46e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:54.386089 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35010710-777d-4732-a858-846e36cb46e1 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:54.406031 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 677/1360] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:43:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:43:56.648033 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cd74d-411a-472f-9b96-004f2872bdfb None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:56.648827 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cd74d-411a-472f-9b96-004f2872bdfb None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:56.648827 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cd74d-411a-472f-9b96-004f2872bdfb None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:56.650202 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 684/1361] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:43:56.655825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01bceb9-250e-42b8-959d-3c647e9ba0fa None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:56.656322 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01bceb9-250e-42b8-959d-3c647e9ba0fa None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:56.656404 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01bceb9-250e-42b8-959d-3c647e9ba0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:56.726549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c01bceb9-250e-42b8-959d-3c647e9ba0fa None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:43:56.727914 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c01bceb9-250e-42b8-959d-3c647e9ba0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:56.778737 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:56.844400 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 678/1362] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:43:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:43:56.889857 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cd74d-411a-472f-9b96-004f2872bdfb None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:56.892670 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-707cd74d-411a-472f-9b96-004f2872bdfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:56.986926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:57.090985 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cd74d-411a-472f-9b96-004f2872bdfb None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:57.240710 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43c5cc-455d-4331-bb4c-bf729e721020 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:57.241468 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43c5cc-455d-4331-bb4c-bf729e721020 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:57.241836 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43c5cc-455d-4331-bb4c-bf729e721020 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:57.242167 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43c5cc-455d-4331-bb4c-bf729e721020 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:57.243865 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7e43c5cc-455d-4331-bb4c-bf729e721020 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:57.268284 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:57.339313 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e43c5cc-455d-4331-bb4c-bf729e721020 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:57.340993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e43c5cc-455d-4331-bb4c-bf729e721020 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:57.361215 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 685/1363] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:43:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:43:59.987434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a3682f-93c5-4567-9ffc-9a19edd5f47a None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:59.988085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a3682f-93c5-4567-9ffc-9a19edd5f47a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:59.988085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a3682f-93c5-4567-9ffc-9a19edd5f47a None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:59.989608 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 679/1364] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:43:59.994673 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb7dc2-4370-4b2c-b2d5-022df3b800c5 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:59.995078 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb7dc2-4370-4b2c-b2d5-022df3b800c5 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:59.995265 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb7dc2-4370-4b2c-b2d5-022df3b800c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:00.075865 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-34fb7dc2-4370-4b2c-b2d5-022df3b800c5 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:44:00.077318 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-34fb7dc2-4370-4b2c-b2d5-022df3b800c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:00.194040 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:00.196854 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 686/1365] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:43:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:44:00.245138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df8227-a75d-4160-bf9f-96825b4a4a3b None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:00.245332 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df8227-a75d-4160-bf9f-96825b4a4a3b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:00.245513 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df8227-a75d-4160-bf9f-96825b4a4a3b None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:00.247165 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 680/1366] 10.222.0.97 () {54 vars in 962 bytes} [Tue Jul 26 14:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:44:00.256778 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925e3d3-e82e-4dca-8397-6c77e504bff9 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:00.257067 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925e3d3-e82e-4dca-8397-6c77e504bff9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:00.257346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925e3d3-e82e-4dca-8397-6c77e504bff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:00.342236 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3925e3d3-e82e-4dca-8397-6c77e504bff9 None None] MFA Rules not processed for user `266abef6cda5495b897170f92043aa03`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:44:00.343519 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3925e3d3-e82e-4dca-8397-6c77e504bff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:00.477603 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 687/1367] 10.222.0.97 () {58 vars in 1059 bytes} [Tue Jul 26 14:44:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:44:00.485521 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df8227-a75d-4160-bf9f-96825b4a4a3b None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:44:00.487032 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-22df8227-a75d-4160-bf9f-96825b4a4a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:00.650589 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df8227-a75d-4160-bf9f-96825b4a4a3b None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:44:00.672735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:00.772915 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5165a8-6cc0-48ef-a32b-dbc955df0ae0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:44:00.773549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5165a8-6cc0-48ef-a32b-dbc955df0ae0 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:00.773837 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5165a8-6cc0-48ef-a32b-dbc955df0ae0 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:00.774148 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5165a8-6cc0-48ef-a32b-dbc955df0ae0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:00.775903 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ca5165a8-6cc0-48ef-a32b-dbc955df0ae0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:00.881956 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5165a8-6cc0-48ef-a32b-dbc955df0ae0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:44:00.883791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5165a8-6cc0-48ef-a32b-dbc955df0ae0 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:44:00.906257 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 681/1368] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:44:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:44:01.784344 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540642cd-d1cd-4844-8113-776aa98bee4a None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:01.784912 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540642cd-d1cd-4844-8113-776aa98bee4a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:01.784912 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540642cd-d1cd-4844-8113-776aa98bee4a None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:01.786728 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 688/1369] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:44:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:44:01.791637 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1568707-ef03-4a69-9131-9ffd55540b54 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:01.792402 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1568707-ef03-4a69-9131-9ffd55540b54 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:01.792923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1568707-ef03-4a69-9131-9ffd55540b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:01.826847 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:01.866219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-d1568707-ef03-4a69-9131-9ffd55540b54 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:44:01.867555 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d1568707-ef03-4a69-9131-9ffd55540b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:01.983453 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 682/1370] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:44:02.343873 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cb1ca-94a0-4cac-8f06-8b0988f9683c None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:02.343873 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cb1ca-94a0-4cac-8f06-8b0988f9683c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:02.343873 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cb1ca-94a0-4cac-8f06-8b0988f9683c None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:02.345152 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 689/1371] 10.222.0.97 () {54 vars in 964 bytes} [Tue Jul 26 14:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:44:02.350229 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf9a98-97fe-4047-97f6-974f4f2e4c27 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:02.350520 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf9a98-97fe-4047-97f6-974f4f2e4c27 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:02.350802 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf9a98-97fe-4047-97f6-974f4f2e4c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:02.435104 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9faf9a98-97fe-4047-97f6-974f4f2e4c27 None None] MFA Rules not processed for user `bad45b307edc415fa6185a82af64ede4`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:44:02.436704 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9faf9a98-97fe-4047-97f6-974f4f2e4c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:02.487483 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:02.520560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 683/1372] 10.222.0.97 () {58 vars in 1061 bytes} [Tue Jul 26 14:44:02 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:44:02.532334 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cb1ca-94a0-4cac-8f06-8b0988f9683c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:44:02.534116 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0d4cb1ca-94a0-4cac-8f06-8b0988f9683c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:02.592775 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:02.650874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cb1ca-94a0-4cac-8f06-8b0988f9683c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:44:02.741644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f02bfe-ff3b-445f-b033-e83552a355cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:44:02.742556 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f02bfe-ff3b-445f-b033-e83552a355cf service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:02.742965 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f02bfe-ff3b-445f-b033-e83552a355cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:02.743310 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f02bfe-ff3b-445f-b033-e83552a355cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:02.745130 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-47f02bfe-ff3b-445f-b033-e83552a355cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:02.853271 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:02.866705 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f02bfe-ff3b-445f-b033-e83552a355cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:44:02.868191 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f02bfe-ff3b-445f-b033-e83552a355cf service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:44:02.888600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 690/1373] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:44:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:44:03.237030 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b4bf8-90e4-4b8f-b717-038d5eab85d6 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:03.237738 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b4bf8-90e4-4b8f-b717-038d5eab85d6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:03.237738 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b4bf8-90e4-4b8f-b717-038d5eab85d6 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:03.239548 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 684/1374] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:44:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:44:03.245277 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ee1d1-b953-4977-9fac-53436d33376a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:03.245572 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ee1d1-b953-4977-9fac-53436d33376a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:03.245854 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ee1d1-b953-4977-9fac-53436d33376a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:03.314136 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-508ee1d1-b953-4977-9fac-53436d33376a None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:44:03.315395 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-508ee1d1-b953-4977-9fac-53436d33376a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:03.412885 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 691/1375] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:44:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:44:03.457112 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b4bf8-90e4-4b8f-b717-038d5eab85d6 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:44:03.458622 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-212b4bf8-90e4-4b8f-b717-038d5eab85d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:03.614031 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b4bf8-90e4-4b8f-b717-038d5eab85d6 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:44:03.665986 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:03.727930 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f060a-b565-4042-84ae-6683fe54dd31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:44:03.728566 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f060a-b565-4042-84ae-6683fe54dd31 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:03.728838 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f060a-b565-4042-84ae-6683fe54dd31 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:03.729115 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f060a-b565-4042-84ae-6683fe54dd31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:03.730751 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-862f060a-b565-4042-84ae-6683fe54dd31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:03.824673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862f060a-b565-4042-84ae-6683fe54dd31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:44:03.826305 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862f060a-b565-4042-84ae-6683fe54dd31 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:44:03.844632 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 685/1376] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:44:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:44:06.107149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2c4bd-c8f4-4b5e-ace9-69f7e14deed7 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:06.107783 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2c4bd-c8f4-4b5e-ace9-69f7e14deed7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:06.107783 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2c4bd-c8f4-4b5e-ace9-69f7e14deed7 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:06.109736 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 692/1377] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:44:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:44:06.115260 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b5b71-4439-43f7-82e0-804f04249be6 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:06.115747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b5b71-4439-43f7-82e0-804f04249be6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:06.115955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b5b71-4439-43f7-82e0-804f04249be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:06.192249 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b50b5b71-4439-43f7-82e0-804f04249be6 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:44:06.193616 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b50b5b71-4439-43f7-82e0-804f04249be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:06.200433 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:06.304030 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 686/1378] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:44:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:44:06.347762 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2c4bd-c8f4-4b5e-ace9-69f7e14deed7 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:44:06.349371 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-03e2c4bd-c8f4-4b5e-ace9-69f7e14deed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:06.413849 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:06.501736 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2c4bd-c8f4-4b5e-ace9-69f7e14deed7 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:44:06.632295 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c2b92-744c-4626-8e89-ea42303d9de6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:44:06.632295 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c2b92-744c-4626-8e89-ea42303d9de6 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:06.632295 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c2b92-744c-4626-8e89-ea42303d9de6 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:06.632295 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c2b92-744c-4626-8e89-ea42303d9de6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:06.633933 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-de1c2b92-744c-4626-8e89-ea42303d9de6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:06.655308 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:06.750562 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c2b92-744c-4626-8e89-ea42303d9de6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:44:06.752330 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c2b92-744c-4626-8e89-ea42303d9de6 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:44:06.774197 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 693/1379] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:44:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:44:09.001029 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8c093-0adf-4d4a-8233-3de4f0c6d24e None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:09.001575 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8c093-0adf-4d4a-8233-3de4f0c6d24e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:09.001575 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8c093-0adf-4d4a-8233-3de4f0c6d24e None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:09.002993 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 687/1380] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:44:09.008298 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b073f71-12fc-4c15-88d6-ae918a2ecdcf None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:09.008994 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b073f71-12fc-4c15-88d6-ae918a2ecdcf None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:09.008994 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b073f71-12fc-4c15-88d6-ae918a2ecdcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:09.088527 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-1b073f71-12fc-4c15-88d6-ae918a2ecdcf None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:44:09.089717 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1b073f71-12fc-4c15-88d6-ae918a2ecdcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:09.191471 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:09.199491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 694/1381] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:44:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:44:09.242525 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace8c093-0adf-4d4a-8233-3de4f0c6d24e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:44:09.244213 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ace8c093-0adf-4d4a-8233-3de4f0c6d24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:09.404980 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:09.417434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace8c093-0adf-4d4a-8233-3de4f0c6d24e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:44:09.541171 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81404ae8-72a8-4e49-af71-2b554fe31333 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:44:09.541794 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81404ae8-72a8-4e49-af71-2b554fe31333 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:09.542078 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81404ae8-72a8-4e49-af71-2b554fe31333 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:09.542351 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81404ae8-72a8-4e49-af71-2b554fe31333 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:09.544019 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-81404ae8-72a8-4e49-af71-2b554fe31333 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:09.660200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81404ae8-72a8-4e49-af71-2b554fe31333 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:44:09.661783 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81404ae8-72a8-4e49-af71-2b554fe31333 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:44:09.676563 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:09.687071 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 688/1382] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:44:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:44:12.033868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc52441-ff44-44f7-9f87-1488878264b0 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:12.034637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc52441-ff44-44f7-9f87-1488878264b0 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:12.034637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc52441-ff44-44f7-9f87-1488878264b0 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:12.035912 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 695/1383] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:44:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:44:12.041955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208a74e-e620-4421-bcbd-7caba015112e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:12.042433 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208a74e-e620-4421-bcbd-7caba015112e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:12.042800 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208a74e-e620-4421-bcbd-7caba015112e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:12.144811 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6208a74e-e620-4421-bcbd-7caba015112e None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:44:12.146493 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6208a74e-e620-4421-bcbd-7caba015112e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:12.285388 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 689/1384] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:44:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:44:12.327099 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc52441-ff44-44f7-9f87-1488878264b0 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:44:12.328859 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7bc52441-ff44-44f7-9f87-1488878264b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:12.501966 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc52441-ff44-44f7-9f87-1488878264b0 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:44:12.577878 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:12.626877 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39c907-3ba0-40ab-8ac4-691596a40af8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:44:12.627546 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39c907-3ba0-40ab-8ac4-691596a40af8 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:12.627875 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39c907-3ba0-40ab-8ac4-691596a40af8 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:12.628286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39c907-3ba0-40ab-8ac4-691596a40af8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:12.629967 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ab39c907-3ba0-40ab-8ac4-691596a40af8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:12.726873 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab39c907-3ba0-40ab-8ac4-691596a40af8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:44:12.729328 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab39c907-3ba0-40ab-8ac4-691596a40af8 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:44:12.758358 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 696/1385] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:44:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:44:14.930683 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed9941-66d2-4b33-8661-045be2e65ddc None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:14.931192 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed9941-66d2-4b33-8661-045be2e65ddc None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:14.931333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed9941-66d2-4b33-8661-045be2e65ddc None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:14.933198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 690/1386] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:44:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:44:14.939360 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d21c1-794d-49af-94c2-c5c9c830db72 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:14.939876 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d21c1-794d-49af-94c2-c5c9c830db72 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:14.939981 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d21c1-794d-49af-94c2-c5c9c830db72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:15.020042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ff1d21c1-794d-49af-94c2-c5c9c830db72 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:44:15.021272 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ff1d21c1-794d-49af-94c2-c5c9c830db72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:15.059644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:15.134592 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 697/1387] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:44:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:44:15.185278 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed9941-66d2-4b33-8661-045be2e65ddc None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:44:15.187797 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-73ed9941-66d2-4b33-8661-045be2e65ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:15.282292 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:15.367946 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed9941-66d2-4b33-8661-045be2e65ddc None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:44:15.502010 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e950b71-fdb2-41fe-ba06-26341eb1610d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:44:15.502487 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e950b71-fdb2-41fe-ba06-26341eb1610d service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:15.502767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e950b71-fdb2-41fe-ba06-26341eb1610d service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:15.503044 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e950b71-fdb2-41fe-ba06-26341eb1610d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:15.504762 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0e950b71-fdb2-41fe-ba06-26341eb1610d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:15.555424 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:15.619878 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e950b71-fdb2-41fe-ba06-26341eb1610d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:44:15.621549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e950b71-fdb2-41fe-ba06-26341eb1610d service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:44:15.646031 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 691/1388] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:44:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:44:17.860611 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed104f-f0f5-4abc-94e0-c540655edd0f None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:17.861135 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed104f-f0f5-4abc-94e0-c540655edd0f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:17.861218 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed104f-f0f5-4abc-94e0-c540655edd0f None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:17.862583 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 698/1389] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:44:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:44:17.868387 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37a054-bcd6-4497-9f82-a948eccde941 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:17.868831 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37a054-bcd6-4497-9f82-a948eccde941 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:17.868939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37a054-bcd6-4497-9f82-a948eccde941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:17.946765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ee37a054-bcd6-4497-9f82-a948eccde941 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:44:17.948092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ee37a054-bcd6-4497-9f82-a948eccde941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:18.056219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 692/1390] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:44:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:44:18.100959 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed104f-f0f5-4abc-94e0-c540655edd0f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:44:18.102458 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a1ed104f-f0f5-4abc-94e0-c540655edd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:18.255017 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed104f-f0f5-4abc-94e0-c540655edd0f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:44:18.272623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:18.376585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686caef0-3ef7-4e53-add8-337320289789 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:44:18.377115 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686caef0-3ef7-4e53-add8-337320289789 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:18.377397 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686caef0-3ef7-4e53-add8-337320289789 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:18.377673 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686caef0-3ef7-4e53-add8-337320289789 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:18.379321 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-686caef0-3ef7-4e53-add8-337320289789 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:18.471541 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686caef0-3ef7-4e53-add8-337320289789 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:44:18.473190 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686caef0-3ef7-4e53-add8-337320289789 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:44:18.491012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 699/1391] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:44:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:44:18.695509 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e878cf-3356-4513-aefb-cff62a76b60f None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:18.695832 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e878cf-3356-4513-aefb-cff62a76b60f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:18.696117 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e878cf-3356-4513-aefb-cff62a76b60f None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:18.697586 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 693/1392] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:44:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:44:18.703815 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e9cf2-b152-4eed-8d2f-847597fcdef1 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:18.704392 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e9cf2-b152-4eed-8d2f-847597fcdef1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:18.704666 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e9cf2-b152-4eed-8d2f-847597fcdef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:18.730574 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:18.772595 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-835e9cf2-b152-4eed-8d2f-847597fcdef1 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:44:18.773878 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-835e9cf2-b152-4eed-8d2f-847597fcdef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:18.872354 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 700/1393] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:44:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:44:18.932079 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91ed32-7d37-40c3-9d10-1c0c0ab826f4 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:18.932686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91ed32-7d37-40c3-9d10-1c0c0ab826f4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:18.932800 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91ed32-7d37-40c3-9d10-1c0c0ab826f4 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:18.934533 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 694/1394] 10.222.0.97 () {54 vars in 966 bytes} [Tue Jul 26 14:44:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:44:18.939970 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf21b8-60d2-4261-a840-72f30aac32a1 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:18.940262 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf21b8-60d2-4261-a840-72f30aac32a1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:18.940546 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf21b8-60d2-4261-a840-72f30aac32a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:19.028060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-bbcf21b8-60d2-4261-a840-72f30aac32a1 None None] MFA Rules not processed for user `62adb1f5e9654927847e1717630ff6b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:44:19.029399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-bbcf21b8-60d2-4261-a840-72f30aac32a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:19.076036 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:19.114649 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 701/1395] 10.222.0.97 () {58 vars in 1063 bytes} [Tue Jul 26 14:44:18 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:44:19.122180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac91ed32-7d37-40c3-9d10-1c0c0ab826f4 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:44:19.124205 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ac91ed32-7d37-40c3-9d10-1c0c0ab826f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:19.138150 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:19.231749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac91ed32-7d37-40c3-9d10-1c0c0ab826f4 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:44:19.308770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6fcb8-c7ea-46a3-a029-c1aee9806084 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62adb1f5e9654927847e1717630ff6b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:44:19.309316 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6fcb8-c7ea-46a3-a029-c1aee9806084 service glance] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:19.309608 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6fcb8-c7ea-46a3-a029-c1aee9806084 service glance] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:19.309884 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6fcb8-c7ea-46a3-a029-c1aee9806084 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:19.311568 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f7e6fcb8-c7ea-46a3-a029-c1aee9806084 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:19.383202 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:19.428899 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e6fcb8-c7ea-46a3-a029-c1aee9806084 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:44:19.430338 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e6fcb8-c7ea-46a3-a029-c1aee9806084 service glance] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:44:19.449870 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 695/1396] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:44:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:44:20.422752 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28952e7-ac52-489f-95c7-30e2eb2be156 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:20.423372 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28952e7-ac52-489f-95c7-30e2eb2be156 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:20.423372 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28952e7-ac52-489f-95c7-30e2eb2be156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:20.499354 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f28952e7-ac52-489f-95c7-30e2eb2be156 None None] MFA Rules not processed for user `5179c3c4eaaa41f3beb79ba7f1993478`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:44:20.500593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f28952e7-ac52-489f-95c7-30e2eb2be156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:20.590869 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 702/1397] 10.222.0.97 () {58 vars in 1009 bytes} [Tue Jul 26 14:44:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:44:20.633885 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e54a6-b3e0-4d9c-adbb-184bc8d214c8 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:20.634368 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e54a6-b3e0-4d9c-adbb-184bc8d214c8 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:20.634572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e54a6-b3e0-4d9c-adbb-184bc8d214c8 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:20.636151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 696/1398] 10.222.0.97 () {54 vars in 943 bytes} [Tue Jul 26 14:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:44:20.643768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730bec1-7180-4d9f-99d6-a260abf7f9ed None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:20.644063 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730bec1-7180-4d9f-99d6-a260abf7f9ed None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:20.644342 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730bec1-7180-4d9f-99d6-a260abf7f9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:20.715725 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:20.723818 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-9730bec1-7180-4d9f-99d6-a260abf7f9ed None None] MFA Rules not processed for user `85122c4052fd4ba19a2dae185a707caf`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:44:20.725215 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9730bec1-7180-4d9f-99d6-a260abf7f9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:20.835082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 703/1399] 10.222.0.97 () {58 vars in 1040 bytes} [Tue Jul 26 14:44:20 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:44:20.844639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e54a6-b3e0-4d9c-adbb-184bc8d214c8 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:44:20.846195 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-bc3e54a6-b3e0-4d9c-adbb-184bc8d214c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:20.944847 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08676874-067a-40cd-a386-e524e80e102a None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:20.945133 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08676874-067a-40cd-a386-e524e80e102a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:20.945409 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08676874-067a-40cd-a386-e524e80e102a None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:20.946763 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 704/1400] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:44:20.952603 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9a10b-7ad0-4869-8156-d67c3b854902 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:20.952887 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9a10b-7ad0-4869-8156-d67c3b854902 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:20.953169 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9a10b-7ad0-4869-8156-d67c3b854902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:20.988605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e54a6-b3e0-4d9c-adbb-184bc8d214c8 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:44:21.023616 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d3f9a10b-7ad0-4869-8156-d67c3b854902 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:44:21.024854 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d3f9a10b-7ad0-4869-8156-d67c3b854902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:21.030816 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:21.089184 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:21.111089 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980eea4-b15c-4786-969f-3cd255847f96 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85122c4052fd4ba19a2dae185a707caf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:44:21.111751 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980eea4-b15c-4786-969f-3cd255847f96 service swift] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:21.112058 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980eea4-b15c-4786-969f-3cd255847f96 service swift] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:21.112329 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980eea4-b15c-4786-969f-3cd255847f96 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:21.114106 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d980eea4-b15c-4786-969f-3cd255847f96 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:21.131710 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 705/1401] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:44:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:44:21.172141 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9a10b-7ad0-4869-8156-d67c3b854902 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:44:21.173370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d3f9a10b-7ad0-4869-8156-d67c3b854902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:21.203374 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d980eea4-b15c-4786-969f-3cd255847f96 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:44:21.205089 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d980eea4-b15c-4786-969f-3cd255847f96 service swift] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:44:21.213783 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 697/1402] 10.222.0.97 () {60 vars in 1384 bytes} [Tue Jul 26 14:44:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 14:44:21.290866 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9a10b-7ad0-4869-8156-d67c3b854902 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:44:21.368454 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:21.417387 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d3fe0-c209-4f45-aff1-a014281f2240 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:44:21.418129 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d3fe0-c209-4f45-aff1-a014281f2240 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:21.418452 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d3fe0-c209-4f45-aff1-a014281f2240 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:21.418752 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d3fe0-c209-4f45-aff1-a014281f2240 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:21.420713 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b25d3fe0-c209-4f45-aff1-a014281f2240 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:21.516920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d3fe0-c209-4f45-aff1-a014281f2240 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:44:21.518551 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d3fe0-c209-4f45-aff1-a014281f2240 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:44:21.537439 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 706/1403] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:44:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:44:24.141210 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b74d9-ff47-45b0-8cf9-e5ff31b970b2 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:24.141833 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b74d9-ff47-45b0-8cf9-e5ff31b970b2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:24.141833 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b74d9-ff47-45b0-8cf9-e5ff31b970b2 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:24.143435 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 698/1404] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:44:24.148970 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227c5bd-87d2-41b8-b54c-6a21586827b3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:24.149468 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227c5bd-87d2-41b8-b54c-6a21586827b3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:24.149551 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227c5bd-87d2-41b8-b54c-6a21586827b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:24.232862 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d227c5bd-87d2-41b8-b54c-6a21586827b3 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:44:24.234094 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d227c5bd-87d2-41b8-b54c-6a21586827b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:24.285835 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:24.353505 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 707/1405] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:44:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:44:24.402513 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b74d9-ff47-45b0-8cf9-e5ff31b970b2 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:44:24.404375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3b4b74d9-ff47-45b0-8cf9-e5ff31b970b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:24.498229 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:24.583130 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b74d9-ff47-45b0-8cf9-e5ff31b970b2 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:44:24.726868 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae65800-dc24-4f0c-8c6a-49ee08ccd81b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:44:24.727522 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae65800-dc24-4f0c-8c6a-49ee08ccd81b service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:24.727809 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae65800-dc24-4f0c-8c6a-49ee08ccd81b service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:24.728087 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae65800-dc24-4f0c-8c6a-49ee08ccd81b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:24.729724 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-eae65800-dc24-4f0c-8c6a-49ee08ccd81b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:24.767645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:24.864236 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae65800-dc24-4f0c-8c6a-49ee08ccd81b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:44:24.864236 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae65800-dc24-4f0c-8c6a-49ee08ccd81b service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:44:24.872229 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 699/1406] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:44:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:44:27.973333 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409dcfb8-48a8-4bfd-9878-75afeab80a21 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:27.973918 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409dcfb8-48a8-4bfd-9878-75afeab80a21 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:27.973918 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409dcfb8-48a8-4bfd-9878-75afeab80a21 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:27.975381 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 708/1407] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:44:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:44:27.980639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef25fd-fecc-4d3e-8fc0-27112eb26530 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:27.981181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef25fd-fecc-4d3e-8fc0-27112eb26530 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:27.981181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef25fd-fecc-4d3e-8fc0-27112eb26530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:28.055290 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e0ef25fd-fecc-4d3e-8fc0-27112eb26530 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:44:28.056554 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e0ef25fd-fecc-4d3e-8fc0-27112eb26530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:28.170567 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 700/1408] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:44:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:44:28.215992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409dcfb8-48a8-4bfd-9878-75afeab80a21 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:44:28.219186 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-409dcfb8-48a8-4bfd-9878-75afeab80a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:28.384180 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409dcfb8-48a8-4bfd-9878-75afeab80a21 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:44:28.408352 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:28.511018 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5c6d9-b764-4191-b773-2c7b5f5ae92e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:44:28.511672 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5c6d9-b764-4191-b773-2c7b5f5ae92e service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:28.511962 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5c6d9-b764-4191-b773-2c7b5f5ae92e service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:28.512237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5c6d9-b764-4191-b773-2c7b5f5ae92e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:28.514064 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-81f5c6d9-b764-4191-b773-2c7b5f5ae92e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:28.629094 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f5c6d9-b764-4191-b773-2c7b5f5ae92e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:44:28.631017 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f5c6d9-b764-4191-b773-2c7b5f5ae92e service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:44:28.653914 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 709/1409] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:44:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:44:38.168869 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:38.648916 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=88 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:45.695605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31aa36b-7a33-4d6b-9220-2e876fd3cb53 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:45.696156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31aa36b-7a33-4d6b-9220-2e876fd3cb53 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:45.696156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31aa36b-7a33-4d6b-9220-2e876fd3cb53 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:45.697444 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 701/1410] 10.222.0.97 () {54 vars in 918 bytes} [Tue Jul 26 14:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:44:45.703173 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ded005-c8bc-45dc-b530-b25992985d87 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:45.703637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ded005-c8bc-45dc-b530-b25992985d87 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:45.703752 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ded005-c8bc-45dc-b530-b25992985d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:45.780296 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-40ded005-c8bc-45dc-b530-b25992985d87 None None] MFA Rules not processed for user `252c2d2939134416a3e8efb254f03c5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:44:45.781501 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-40ded005-c8bc-45dc-b530-b25992985d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:46.011785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 710/1411] 10.222.0.97 () {58 vars in 1015 bytes} [Tue Jul 26 14:44:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:44:46.026101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35803e4-393d-473b-92e3-203f6942e76d None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:46.026390 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35803e4-393d-473b-92e3-203f6942e76d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:46.026670 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35803e4-393d-473b-92e3-203f6942e76d None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:46.028591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 702/1412] 10.222.0.97 () {54 vars in 918 bytes} [Tue Jul 26 14:44:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:44:46.033216 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f41ef-cd05-4a05-b1d8-8b93b917f405 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:46.033545 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f41ef-cd05-4a05-b1d8-8b93b917f405 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:46.033917 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f41ef-cd05-4a05-b1d8-8b93b917f405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:46.105763 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:46.109278 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-9b6f41ef-cd05-4a05-b1d8-8b93b917f405 None None] MFA Rules not processed for user `266abef6cda5495b897170f92043aa03`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:44:46.110474 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9b6f41ef-cd05-4a05-b1d8-8b93b917f405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:46.225134 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 711/1413] 10.222.0.97 () {58 vars in 1015 bytes} [Tue Jul 26 14:44:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:44:48.708758 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756f17c-ffec-41f8-bbe1-d67af61c91d8 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:48.709332 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756f17c-ffec-41f8-bbe1-d67af61c91d8 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:48.709332 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756f17c-ffec-41f8-bbe1-d67af61c91d8 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:48.710652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 703/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 14:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:44:48.716430 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af423f47-83b5-4828-b881-b0e03de04e8e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:48.716822 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af423f47-83b5-4828-b881-b0e03de04e8e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:48.716967 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af423f47-83b5-4828-b881-b0e03de04e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:48.783722 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-af423f47-83b5-4828-b881-b0e03de04e8e None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:44:48.784935 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-af423f47-83b5-4828-b881-b0e03de04e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:48.840326 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:48.882498 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 712/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 14:44:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:44:48.925421 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756f17c-ffec-41f8-bbe1-d67af61c91d8 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:44:48.926928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e756f17c-ffec-41f8-bbe1-d67af61c91d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:49.072114 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756f17c-ffec-41f8-bbe1-d67af61c91d8 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:44:49.155894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:49.197027 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02ed1f-3678-4f10-b26d-3b9f07039aef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:44:49.197641 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02ed1f-3678-4f10-b26d-3b9f07039aef service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:49.197918 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02ed1f-3678-4f10-b26d-3b9f07039aef service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:49.198187 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02ed1f-3678-4f10-b26d-3b9f07039aef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:49.200456 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1d02ed1f-3678-4f10-b26d-3b9f07039aef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:49.289270 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d02ed1f-3678-4f10-b26d-3b9f07039aef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:44:49.290974 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d02ed1f-3678-4f10-b26d-3b9f07039aef service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:44:49.310760 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 704/1416] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:44:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:44:50.207687 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0812fa7-7269-470d-b28c-3097fd521a8a None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:50.208460 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0812fa7-7269-470d-b28c-3097fd521a8a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:50.208854 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0812fa7-7269-470d-b28c-3097fd521a8a None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:50.210312 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 713/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 14:44:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:44:50.215088 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ea219-5c69-4af0-999c-ff9528f282f3 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:50.216833 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ea219-5c69-4af0-999c-ff9528f282f3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:50.216833 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ea219-5c69-4af0-999c-ff9528f282f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:50.292425 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-996ea219-5c69-4af0-999c-ff9528f282f3 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:44:50.293624 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-996ea219-5c69-4af0-999c-ff9528f282f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:50.320184 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:50.391472 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 705/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 14:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:44:50.542090 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0812fa7-7269-470d-b28c-3097fd521a8a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:44:50.544256 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d0812fa7-7269-470d-b28c-3097fd521a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:50.689570 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0812fa7-7269-470d-b28c-3097fd521a8a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:44:50.729093 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:50.803575 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757785a-8b53-4682-a797-daa61a904bde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:44:50.804194 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757785a-8b53-4682-a797-daa61a904bde service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:50.804482 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757785a-8b53-4682-a797-daa61a904bde service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:50.804760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757785a-8b53-4682-a797-daa61a904bde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:50.806388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8757785a-8b53-4682-a797-daa61a904bde service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:50.896599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8757785a-8b53-4682-a797-daa61a904bde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:44:50.898184 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8757785a-8b53-4682-a797-daa61a904bde service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:44:50.915640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 714/1419] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:44:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:44:51.811247 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7debb4a6-5e29-486b-a1e9-1464b20f4853 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:51.811869 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7debb4a6-5e29-486b-a1e9-1464b20f4853 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:51.811869 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7debb4a6-5e29-486b-a1e9-1464b20f4853 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:51.813454 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 706/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 14:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:44:51.818535 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ada9b3-d932-44e3-ae47-3c71977ad55c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:51.819005 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ada9b3-d932-44e3-ae47-3c71977ad55c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:51.819103 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ada9b3-d932-44e3-ae47-3c71977ad55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:51.873171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:51.896637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-74ada9b3-d932-44e3-ae47-3c71977ad55c None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:44:51.897884 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-74ada9b3-d932-44e3-ae47-3c71977ad55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:52.007003 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 715/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 14:44:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:44:52.160819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7debb4a6-5e29-486b-a1e9-1464b20f4853 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:44:52.162357 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7debb4a6-5e29-486b-a1e9-1464b20f4853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:52.311918 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7debb4a6-5e29-486b-a1e9-1464b20f4853 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:44:52.324445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:52.424781 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975c456-b7a2-438b-b7df-92c858dfbe5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:44:52.425313 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975c456-b7a2-438b-b7df-92c858dfbe5c service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:52.425587 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975c456-b7a2-438b-b7df-92c858dfbe5c service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:52.425866 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975c456-b7a2-438b-b7df-92c858dfbe5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:52.427479 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1975c456-b7a2-438b-b7df-92c858dfbe5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:52.517130 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975c456-b7a2-438b-b7df-92c858dfbe5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:44:52.518749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975c456-b7a2-438b-b7df-92c858dfbe5c service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:44:52.536921 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 707/1422] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:44:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:44:53.401679 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c2637-f967-4306-9c84-00d3d8cf954a None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:53.402210 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c2637-f967-4306-9c84-00d3d8cf954a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:53.402210 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c2637-f967-4306-9c84-00d3d8cf954a None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:53.404151 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 716/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 14:44:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:44:53.408971 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c5e54-6d3b-42ab-b0e1-f5632d7418b3 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:53.409434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c5e54-6d3b-42ab-b0e1-f5632d7418b3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:53.409538 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c5e54-6d3b-42ab-b0e1-f5632d7418b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:53.429337 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:53.487227 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-816c5e54-6d3b-42ab-b0e1-f5632d7418b3 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:44:53.488476 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-816c5e54-6d3b-42ab-b0e1-f5632d7418b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:53.586814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 708/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 14:44:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:44:53.740341 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c2637-f967-4306-9c84-00d3d8cf954a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:44:53.741772 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1e0c2637-f967-4306-9c84-00d3d8cf954a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:53.851712 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:53.888503 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c2637-f967-4306-9c84-00d3d8cf954a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:44:54.001582 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f029ce17-b5a6-4cad-ac2e-ce837aead74a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:44:54.002116 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029ce17-b5a6-4cad-ac2e-ce837aead74a service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:54.002396 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029ce17-b5a6-4cad-ac2e-ce837aead74a service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:54.002671 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029ce17-b5a6-4cad-ac2e-ce837aead74a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:54.004581 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f029ce17-b5a6-4cad-ac2e-ce837aead74a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:54.082337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:54.096460 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f029ce17-b5a6-4cad-ac2e-ce837aead74a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:44:54.098059 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f029ce17-b5a6-4cad-ac2e-ce837aead74a service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:44:54.115894 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 717/1425] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:44:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:44:54.998347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce9aec-f4b7-4db4-bb52-4824a6f7cb0f None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:54.998871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce9aec-f4b7-4db4-bb52-4824a6f7cb0f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:54.998871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce9aec-f4b7-4db4-bb52-4824a6f7cb0f None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:55.000223 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 709/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 14:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:44:55.005308 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a803c3e-3307-486c-8f65-c5f920ebba48 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:55.005691 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a803c3e-3307-486c-8f65-c5f920ebba48 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:55.005881 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a803c3e-3307-486c-8f65-c5f920ebba48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:55.084253 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-7a803c3e-3307-486c-8f65-c5f920ebba48 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:44:55.085454 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7a803c3e-3307-486c-8f65-c5f920ebba48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:55.182183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 718/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 14:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:44:55.337441 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce9aec-f4b7-4db4-bb52-4824a6f7cb0f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:44:55.339130 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-40ce9aec-f4b7-4db4-bb52-4824a6f7cb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:55.426864 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:55.489616 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce9aec-f4b7-4db4-bb52-4824a6f7cb0f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:44:55.603549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b532e8d4-851e-4249-a7fa-cee7451162df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:44:55.604136 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532e8d4-851e-4249-a7fa-cee7451162df service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:55.604425 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532e8d4-851e-4249-a7fa-cee7451162df service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:55.604704 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532e8d4-851e-4249-a7fa-cee7451162df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:55.606870 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b532e8d4-851e-4249-a7fa-cee7451162df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:55.696157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b532e8d4-851e-4249-a7fa-cee7451162df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:44:55.697773 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b532e8d4-851e-4249-a7fa-cee7451162df service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:44:55.703981 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:55.717615 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 710/1428] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:44:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:44:56.695652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa9974-24cf-4979-bb0d-c3b9aaf001f2 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:56.696304 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa9974-24cf-4979-bb0d-c3b9aaf001f2 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:56.696304 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa9974-24cf-4979-bb0d-c3b9aaf001f2 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:56.697584 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 719/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 14:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:44:56.702969 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d479e-84ca-4720-9dff-a74a70fc6f1d None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:56.703452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d479e-84ca-4720-9dff-a74a70fc6f1d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:56.703557 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d479e-84ca-4720-9dff-a74a70fc6f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:56.784467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-828d479e-84ca-4720-9dff-a74a70fc6f1d None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:44:56.785718 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-828d479e-84ca-4720-9dff-a74a70fc6f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:56.885418 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 711/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 14:44:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:44:57.039067 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aa9974-24cf-4979-bb0d-c3b9aaf001f2 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:44:57.041820 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-20aa9974-24cf-4979-bb0d-c3b9aaf001f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:57.092332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:57.191134 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aa9974-24cf-4979-bb0d-c3b9aaf001f2 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:44:57.306292 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f963b09-9df0-46b0-81d5-c37f2bc16d86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:44:57.306827 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f963b09-9df0-46b0-81d5-c37f2bc16d86 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:57.307106 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f963b09-9df0-46b0-81d5-c37f2bc16d86 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:57.307376 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f963b09-9df0-46b0-81d5-c37f2bc16d86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:57.309570 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0f963b09-9df0-46b0-81d5-c37f2bc16d86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:57.316706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:57.399622 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f963b09-9df0-46b0-81d5-c37f2bc16d86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:44:57.401219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f963b09-9df0-46b0-81d5-c37f2bc16d86 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:44:57.419550 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 720/1431] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:44:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:44:58.288377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b99fa-0b91-4819-8c55-f2eb884d78f3 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:58.289136 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b99fa-0b91-4819-8c55-f2eb884d78f3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:58.289136 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b99fa-0b91-4819-8c55-f2eb884d78f3 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:58.290393 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 712/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 14:44:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:44:58.295556 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ec3db-3e04-43e5-a736-04f638ab05e2 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:58.295984 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ec3db-3e04-43e5-a736-04f638ab05e2 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:58.296094 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ec3db-3e04-43e5-a736-04f638ab05e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:58.376868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ce4ec3db-3e04-43e5-a736-04f638ab05e2 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:44:58.377998 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ce4ec3db-3e04-43e5-a736-04f638ab05e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:58.447141 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:58.476532 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 721/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 14:44:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:44:58.629721 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b99fa-0b91-4819-8c55-f2eb884d78f3 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:44:58.631191 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-956b99fa-0b91-4819-8c55-f2eb884d78f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:58.649802 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:58.781833 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b99fa-0b91-4819-8c55-f2eb884d78f3 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:44:58.868694 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:44:58.895290 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7ddac-76a9-466e-b61e-eac7311c4766 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:44:58.895879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7ddac-76a9-466e-b61e-eac7311c4766 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:58.896165 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7ddac-76a9-466e-b61e-eac7311c4766 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:58.896445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7ddac-76a9-466e-b61e-eac7311c4766 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:58.898076 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ded7ddac-76a9-466e-b61e-eac7311c4766 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:44:58.990447 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded7ddac-76a9-466e-b61e-eac7311c4766 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:44:58.992125 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded7ddac-76a9-466e-b61e-eac7311c4766 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:44:59.010386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 713/1434] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:44:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:44:59.885809 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dfb83a-6665-448a-80eb-47bce22816ca None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:59.886973 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dfb83a-6665-448a-80eb-47bce22816ca None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:59.886973 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dfb83a-6665-448a-80eb-47bce22816ca None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:59.888210 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 722/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 14:44:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:44:59.893172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851cdb2-fc5b-47aa-9050-2b3d401d2cc4 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:44:59.893608 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851cdb2-fc5b-47aa-9050-2b3d401d2cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:44:59.893737 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851cdb2-fc5b-47aa-9050-2b3d401d2cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:44:59.972578 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3851cdb2-fc5b-47aa-9050-2b3d401d2cc4 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:44:59.973793 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3851cdb2-fc5b-47aa-9050-2b3d401d2cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:00.017635 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:00.075453 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 714/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 14:44:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:45:00.226969 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dfb83a-6665-448a-80eb-47bce22816ca None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:00.228418 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b1dfb83a-6665-448a-80eb-47bce22816ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:00.380254 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dfb83a-6665-448a-80eb-47bce22816ca None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:00.441852 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:00.500081 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f141a9a8-5a08-43a8-b4de-e2a40c4faf1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:00.500625 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141a9a8-5a08-43a8-b4de-e2a40c4faf1d service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:00.500909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141a9a8-5a08-43a8-b4de-e2a40c4faf1d service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:00.501184 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141a9a8-5a08-43a8-b4de-e2a40c4faf1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:00.502805 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f141a9a8-5a08-43a8-b4de-e2a40c4faf1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:00.600479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f141a9a8-5a08-43a8-b4de-e2a40c4faf1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:00.602055 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f141a9a8-5a08-43a8-b4de-e2a40c4faf1d service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:00.621509 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 723/1437] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:45:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:45:01.554257 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafbedc-6a29-42d2-a623-3289fbbde821 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:01.554841 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafbedc-6a29-42d2-a623-3289fbbde821 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:01.554841 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafbedc-6a29-42d2-a623-3289fbbde821 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:01.556283 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 715/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 14:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:45:01.561426 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df28a1-1d9a-40fa-bc93-d3569b9f0063 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:01.561867 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df28a1-1d9a-40fa-bc93-d3569b9f0063 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:01.561976 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df28a1-1d9a-40fa-bc93-d3569b9f0063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:01.641479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-31df28a1-1d9a-40fa-bc93-d3569b9f0063 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:45:01.642693 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-31df28a1-1d9a-40fa-bc93-d3569b9f0063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:01.644808 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:01.738687 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 724/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 14:45:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:45:01.889709 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dafbedc-6a29-42d2-a623-3289fbbde821 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:01.891198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6dafbedc-6a29-42d2-a623-3289fbbde821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:02.038595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dafbedc-6a29-42d2-a623-3289fbbde821 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:02.067292 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:02.154110 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36180e62-4a1b-4830-b4b8-240f9fbd1434 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:02.154643 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36180e62-4a1b-4830-b4b8-240f9fbd1434 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:02.154927 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36180e62-4a1b-4830-b4b8-240f9fbd1434 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:02.155204 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36180e62-4a1b-4830-b4b8-240f9fbd1434 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:02.156875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-36180e62-4a1b-4830-b4b8-240f9fbd1434 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:02.247365 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36180e62-4a1b-4830-b4b8-240f9fbd1434 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:02.249010 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36180e62-4a1b-4830-b4b8-240f9fbd1434 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:02.267126 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 716/1440] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:45:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:45:04.174132 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca6a5e-d95f-467c-b603-bd70aa2adfec None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:04.174654 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca6a5e-d95f-467c-b603-bd70aa2adfec None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:04.174769 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca6a5e-d95f-467c-b603-bd70aa2adfec None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:04.176126 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 725/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 14:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:45:04.181741 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64872e48-a1a4-463e-8f52-b349e35fe579 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:04.182167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64872e48-a1a4-463e-8f52-b349e35fe579 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:04.182334 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64872e48-a1a4-463e-8f52-b349e35fe579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:04.223023 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:04.248272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-64872e48-a1a4-463e-8f52-b349e35fe579 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:45:04.249491 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-64872e48-a1a4-463e-8f52-b349e35fe579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:04.348536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 717/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 14:45:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:45:04.387180 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca6a5e-d95f-467c-b603-bd70aa2adfec None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:04.388799 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3aca6a5e-d95f-467c-b603-bd70aa2adfec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:04.491651 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca6a5e-d95f-467c-b603-bd70aa2adfec None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:04.530921 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:04.572393 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af6b9c-f9e7-4453-8243-7cdf887b9f8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:04.573373 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af6b9c-f9e7-4453-8243-7cdf887b9f8b service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:04.574220 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af6b9c-f9e7-4453-8243-7cdf887b9f8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:04.574520 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af6b9c-f9e7-4453-8243-7cdf887b9f8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:04.576551 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-29af6b9c-f9e7-4453-8243-7cdf887b9f8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:04.684618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af6b9c-f9e7-4453-8243-7cdf887b9f8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:04.685993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af6b9c-f9e7-4453-8243-7cdf887b9f8b service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:04.705255 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 726/1443] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:45:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:45:06.913639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a22030-cb8f-4cf7-a92b-ea2c765be6ca None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:06.914194 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a22030-cb8f-4cf7-a92b-ea2c765be6ca None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:06.914194 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a22030-cb8f-4cf7-a92b-ea2c765be6ca None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:06.915591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 718/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 14:45:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:45:06.921049 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b72ee-d5f2-4fd4-80db-f6a7cc4bf15d None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:06.921465 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b72ee-d5f2-4fd4-80db-f6a7cc4bf15d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:06.921636 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b72ee-d5f2-4fd4-80db-f6a7cc4bf15d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:06.987741 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-688b72ee-d5f2-4fd4-80db-f6a7cc4bf15d None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:45:06.988972 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-688b72ee-d5f2-4fd4-80db-f6a7cc4bf15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:07.002705 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:07.090528 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 727/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 14:45:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:45:07.129289 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a22030-cb8f-4cf7-a92b-ea2c765be6ca None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:07.131731 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f1a22030-cb8f-4cf7-a92b-ea2c765be6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:07.238740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a22030-cb8f-4cf7-a92b-ea2c765be6ca None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:07.250965 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:07.315657 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02332e81-b9d1-4e2d-8d7b-9aab9cbb35cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:07.316286 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02332e81-b9d1-4e2d-8d7b-9aab9cbb35cd service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:07.316570 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02332e81-b9d1-4e2d-8d7b-9aab9cbb35cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:07.316847 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02332e81-b9d1-4e2d-8d7b-9aab9cbb35cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:07.318469 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-02332e81-b9d1-4e2d-8d7b-9aab9cbb35cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:07.429537 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02332e81-b9d1-4e2d-8d7b-9aab9cbb35cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:07.430970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02332e81-b9d1-4e2d-8d7b-9aab9cbb35cd service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:07.449658 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 719/1446] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:45:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:45:09.871529 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3bbf7-707d-481b-b56e-3e3e4849a9b1 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:09.872066 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3bbf7-707d-481b-b56e-3e3e4849a9b1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:09.872066 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3bbf7-707d-481b-b56e-3e3e4849a9b1 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:09.873439 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 728/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 14:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:45:09.878884 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82becc3-20dd-42e9-b508-d1a308cea691 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:09.879312 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82becc3-20dd-42e9-b508-d1a308cea691 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:09.879491 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82becc3-20dd-42e9-b508-d1a308cea691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:09.930680 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:09.949491 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a82becc3-20dd-42e9-b508-d1a308cea691 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:45:09.950711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a82becc3-20dd-42e9-b508-d1a308cea691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:10.049786 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 720/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 14:45:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:45:10.088255 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3bbf7-707d-481b-b56e-3e3e4849a9b1 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:10.090285 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f3f3bbf7-707d-481b-b56e-3e3e4849a9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:10.197637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3bbf7-707d-481b-b56e-3e3e4849a9b1 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:10.241404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:10.275487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1d3f1-143b-4a90-b7a9-b52a2ec409d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:10.276046 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1d3f1-143b-4a90-b7a9-b52a2ec409d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:10.277008 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1d3f1-143b-4a90-b7a9-b52a2ec409d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:10.277287 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1d3f1-143b-4a90-b7a9-b52a2ec409d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:10.278905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-78c1d3f1-143b-4a90-b7a9-b52a2ec409d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:10.387999 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1d3f1-143b-4a90-b7a9-b52a2ec409d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:10.389435 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1d3f1-143b-4a90-b7a9-b52a2ec409d2 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:10.408107 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 729/1449] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:45:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:45:12.572158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100216e0-ce1e-492b-aa88-1443d91c6e2b None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:12.572727 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100216e0-ce1e-492b-aa88-1443d91c6e2b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:12.572727 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100216e0-ce1e-492b-aa88-1443d91c6e2b None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:12.574117 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 721/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 14:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:45:12.579439 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78d8f7-d306-42b4-9d4e-5dc9bd068286 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:12.579876 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78d8f7-d306-42b4-9d4e-5dc9bd068286 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:12.579999 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78d8f7-d306-42b4-9d4e-5dc9bd068286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:12.645668 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-fd78d8f7-d306-42b4-9d4e-5dc9bd068286 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:45:12.646874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fd78d8f7-d306-42b4-9d4e-5dc9bd068286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:12.668126 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:12.744676 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 730/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 14:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:45:12.784977 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100216e0-ce1e-492b-aa88-1443d91c6e2b None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:12.786511 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-100216e0-ce1e-492b-aa88-1443d91c6e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:12.896095 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100216e0-ce1e-492b-aa88-1443d91c6e2b None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:12.916628 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:12.971952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a27b11-cb33-48fe-a1fe-74e3984cb7dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:12.972591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a27b11-cb33-48fe-a1fe-74e3984cb7dd service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:12.973423 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a27b11-cb33-48fe-a1fe-74e3984cb7dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:12.973701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a27b11-cb33-48fe-a1fe-74e3984cb7dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:12.975313 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-38a27b11-cb33-48fe-a1fe-74e3984cb7dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:13.085740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a27b11-cb33-48fe-a1fe-74e3984cb7dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:13.087207 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a27b11-cb33-48fe-a1fe-74e3984cb7dd service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:13.105486 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 722/1452] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:45:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:45:15.212767 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d2a09-cfb3-462c-b5cd-58cb08f1460d None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:15.213364 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d2a09-cfb3-462c-b5cd-58cb08f1460d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:15.213364 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d2a09-cfb3-462c-b5cd-58cb08f1460d None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:15.214718 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 731/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 14:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:45:15.220373 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d458b-859b-475e-9ecc-37afad28279a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:15.233464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d458b-859b-475e-9ecc-37afad28279a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:15.233464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d458b-859b-475e-9ecc-37afad28279a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:15.288673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:15.291385 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-f61d458b-859b-475e-9ecc-37afad28279a None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:45:15.293317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f61d458b-859b-475e-9ecc-37afad28279a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:15.395192 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 723/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 14:45:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:45:15.434805 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069d2a09-cfb3-462c-b5cd-58cb08f1460d None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:15.436384 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-069d2a09-cfb3-462c-b5cd-58cb08f1460d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:15.614477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069d2a09-cfb3-462c-b5cd-58cb08f1460d None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:15.665770 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:15.691304 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad0573-94e2-4f74-928f-443f025489e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:15.691939 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad0573-94e2-4f74-928f-443f025489e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:15.692217 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad0573-94e2-4f74-928f-443f025489e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:15.692490 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad0573-94e2-4f74-928f-443f025489e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:15.694127 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b4ad0573-94e2-4f74-928f-443f025489e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:15.809004 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad0573-94e2-4f74-928f-443f025489e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:15.810431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad0573-94e2-4f74-928f-443f025489e6 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:15.830436 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 732/1455] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:45:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:45:17.926536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df885a7f-a361-4372-929c-59934e01e9fb None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:17.927077 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df885a7f-a361-4372-929c-59934e01e9fb None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:17.927077 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df885a7f-a361-4372-929c-59934e01e9fb None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:17.928550 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 724/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 14:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:45:17.934692 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57c696-f70f-4933-992c-e90f7eb4398e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:17.935186 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57c696-f70f-4933-992c-e90f7eb4398e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:17.935341 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57c696-f70f-4933-992c-e90f7eb4398e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:18.006491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ac57c696-f70f-4933-992c-e90f7eb4398e None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:45:18.007751 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ac57c696-f70f-4933-992c-e90f7eb4398e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:18.038888 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:18.112383 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 733/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 14:45:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:45:18.151300 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df885a7f-a361-4372-929c-59934e01e9fb None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:18.152877 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-df885a7f-a361-4372-929c-59934e01e9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:18.261107 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df885a7f-a361-4372-929c-59934e01e9fb None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:18.291638 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:18.342293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9111359-b317-4339-b0e4-216d360b8f66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:18.342919 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9111359-b317-4339-b0e4-216d360b8f66 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:18.352156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9111359-b317-4339-b0e4-216d360b8f66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:18.352156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9111359-b317-4339-b0e4-216d360b8f66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:18.352156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b9111359-b317-4339-b0e4-216d360b8f66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:18.578071 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9111359-b317-4339-b0e4-216d360b8f66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:18.579532 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9111359-b317-4339-b0e4-216d360b8f66 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:18.603338 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 725/1458] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:45:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:45:20.675694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d759f-2a84-49e9-89c0-2801c42464fb None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:20.676311 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d759f-2a84-49e9-89c0-2801c42464fb None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:20.676567 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d759f-2a84-49e9-89c0-2801c42464fb None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:20.678739 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 734/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 14:45:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:45:20.686814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfb801-2f76-41c7-963f-b07e55d0911a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:20.687316 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfb801-2f76-41c7-963f-b07e55d0911a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:20.687581 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfb801-2f76-41c7-963f-b07e55d0911a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:20.757173 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3dbfb801-2f76-41c7-963f-b07e55d0911a None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:45:20.758392 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3dbfb801-2f76-41c7-963f-b07e55d0911a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:20.768255 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:20.860945 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 726/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 14:45:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:45:20.903452 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d759f-2a84-49e9-89c0-2801c42464fb None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:20.905260 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-564d759f-2a84-49e9-89c0-2801c42464fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:21.012338 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d759f-2a84-49e9-89c0-2801c42464fb None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:21.069720 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:21.089785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf0c2d-a4a8-42d0-b548-3aef7caa4a57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:21.090463 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf0c2d-a4a8-42d0-b548-3aef7caa4a57 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:21.090866 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf0c2d-a4a8-42d0-b548-3aef7caa4a57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:21.091150 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf0c2d-a4a8-42d0-b548-3aef7caa4a57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:21.092991 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9abf0c2d-a4a8-42d0-b548-3aef7caa4a57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:21.205924 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf0c2d-a4a8-42d0-b548-3aef7caa4a57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:21.207347 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf0c2d-a4a8-42d0-b548-3aef7caa4a57 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:21.226359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 735/1461] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:45:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:45:23.293240 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281882d9-213b-4341-b06d-a130b8c31232 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:23.293801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281882d9-213b-4341-b06d-a130b8c31232 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:23.293801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281882d9-213b-4341-b06d-a130b8c31232 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:23.295165 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 727/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 14:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:45:23.300618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce329e2-b2be-4b7e-8cff-315423aea613 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:23.301161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce329e2-b2be-4b7e-8cff-315423aea613 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:23.301670 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce329e2-b2be-4b7e-8cff-315423aea613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:23.368412 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-4ce329e2-b2be-4b7e-8cff-315423aea613 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:45:23.369567 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4ce329e2-b2be-4b7e-8cff-315423aea613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:23.409092 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:23.466251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 736/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 14:45:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:45:23.503609 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281882d9-213b-4341-b06d-a130b8c31232 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:23.505127 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-281882d9-213b-4341-b06d-a130b8c31232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:23.609519 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281882d9-213b-4341-b06d-a130b8c31232 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:23.648396 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:23.685452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab39dc9-4f71-478a-95fb-d7bfe925b722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:23.686072 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab39dc9-4f71-478a-95fb-d7bfe925b722 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:23.686349 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab39dc9-4f71-478a-95fb-d7bfe925b722 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:23.686618 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab39dc9-4f71-478a-95fb-d7bfe925b722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:23.688269 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0ab39dc9-4f71-478a-95fb-d7bfe925b722 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:23.800332 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab39dc9-4f71-478a-95fb-d7bfe925b722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:23.801775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab39dc9-4f71-478a-95fb-d7bfe925b722 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:23.820664 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 728/1464] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:45:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:45:25.900533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a69a8-1401-4a77-a1e0-665e39b0586b None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:25.901071 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a69a8-1401-4a77-a1e0-665e39b0586b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:25.901071 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a69a8-1401-4a77-a1e0-665e39b0586b None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:25.902436 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 737/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 14:45:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:45:25.908000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be27a82-30f1-4088-91b0-41115d234e96 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:25.908400 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be27a82-30f1-4088-91b0-41115d234e96 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:25.908564 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be27a82-30f1-4088-91b0-41115d234e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:25.975898 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6be27a82-30f1-4088-91b0-41115d234e96 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:45:25.977117 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6be27a82-30f1-4088-91b0-41115d234e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:25.995622 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:26.076028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 729/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 14:45:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:45:26.113771 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a69a8-1401-4a77-a1e0-665e39b0586b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:26.115315 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-779a69a8-1401-4a77-a1e0-665e39b0586b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:26.219194 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a69a8-1401-4a77-a1e0-665e39b0586b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:26.288367 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:26.295717 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a781344a-9afe-4832-9045-7dfcdc97e70c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:26.296335 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781344a-9afe-4832-9045-7dfcdc97e70c service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:26.296619 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781344a-9afe-4832-9045-7dfcdc97e70c service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:26.296893 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781344a-9afe-4832-9045-7dfcdc97e70c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:26.298499 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a781344a-9afe-4832-9045-7dfcdc97e70c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:26.413782 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a781344a-9afe-4832-9045-7dfcdc97e70c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:26.415206 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a781344a-9afe-4832-9045-7dfcdc97e70c service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:26.434219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 738/1467] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:45:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:45:28.512072 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52521411-fe50-4c26-b613-4021be4ad876 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:28.512657 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52521411-fe50-4c26-b613-4021be4ad876 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:28.512657 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52521411-fe50-4c26-b613-4021be4ad876 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:28.513946 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 730/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 14:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:45:28.519260 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88240c97-aef5-4123-a272-772d9758b56c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:28.520011 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88240c97-aef5-4123-a272-772d9758b56c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:28.520011 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88240c97-aef5-4123-a272-772d9758b56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:28.590798 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-88240c97-aef5-4123-a272-772d9758b56c None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:45:28.592102 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-88240c97-aef5-4123-a272-772d9758b56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:28.643307 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:28.692413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 739/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 14:45:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:45:28.730390 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52521411-fe50-4c26-b613-4021be4ad876 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:28.732137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-52521411-fe50-4c26-b613-4021be4ad876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:28.839465 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52521411-fe50-4c26-b613-4021be4ad876 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:28.889471 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:28.916006 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124dcd8-bc0a-4903-a077-46794976cb86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:28.917233 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124dcd8-bc0a-4903-a077-46794976cb86 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:28.917552 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124dcd8-bc0a-4903-a077-46794976cb86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:28.917825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124dcd8-bc0a-4903-a077-46794976cb86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:28.919597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2124dcd8-bc0a-4903-a077-46794976cb86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:29.035483 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2124dcd8-bc0a-4903-a077-46794976cb86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:29.040284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2124dcd8-bc0a-4903-a077-46794976cb86 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:29.055985 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 731/1470] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:45:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:45:38.709990 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=22 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:39.053619 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=66 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:48.966402 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f771e3-8b2d-4ced-a0c8-2077101041c6 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:48.966983 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f771e3-8b2d-4ced-a0c8-2077101041c6 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:48.966983 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f771e3-8b2d-4ced-a0c8-2077101041c6 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:48.968382 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 740/1471] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:45:48.973407 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb5da5-c300-4e77-9ba6-4a6dfa15742c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:48.973810 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb5da5-c300-4e77-9ba6-4a6dfa15742c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:48.973960 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb5da5-c300-4e77-9ba6-4a6dfa15742c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:49.041035 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-76cb5da5-c300-4e77-9ba6-4a6dfa15742c None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:45:49.042176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-76cb5da5-c300-4e77-9ba6-4a6dfa15742c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:49.138136 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 732/1472] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:45:49.221824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3140ab-9f48-42ef-a84a-095dadb4e0af None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:49.238049 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3140ab-9f48-42ef-a84a-095dadb4e0af None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:49.238049 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3140ab-9f48-42ef-a84a-095dadb4e0af None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:49.238049 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 741/1473] 10.222.0.97 () {54 vars in 966 bytes} [Tue Jul 26 14:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:45:49.238342 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59edca7d-69b6-4cde-aa3c-97355957e669 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:49.238342 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59edca7d-69b6-4cde-aa3c-97355957e669 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:49.238342 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59edca7d-69b6-4cde-aa3c-97355957e669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:49.306059 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-59edca7d-69b6-4cde-aa3c-97355957e669 None None] MFA Rules not processed for user `62adb1f5e9654927847e1717630ff6b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:45:49.307206 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-59edca7d-69b6-4cde-aa3c-97355957e669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:49.327831 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:49.383805 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 733/1474] 10.222.0.97 () {58 vars in 1063 bytes} [Tue Jul 26 14:45:49 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:45:49.392919 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3140ab-9f48-42ef-a84a-095dadb4e0af None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:49.394689 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7e3140ab-9f48-42ef-a84a-095dadb4e0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:49.499563 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3140ab-9f48-42ef-a84a-095dadb4e0af None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:49.573739 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01acbd-b692-4489-bf69-9729e8c5f346 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62adb1f5e9654927847e1717630ff6b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:49.574281 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01acbd-b692-4489-bf69-9729e8c5f346 service glance] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:49.574570 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01acbd-b692-4489-bf69-9729e8c5f346 service glance] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:49.574846 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01acbd-b692-4489-bf69-9729e8c5f346 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:49.576496 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ae01acbd-b692-4489-bf69-9729e8c5f346 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:49.633088 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:49.688943 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01acbd-b692-4489-bf69-9729e8c5f346 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:49.690378 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01acbd-b692-4489-bf69-9729e8c5f346 service glance] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:49.708858 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 742/1475] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:45:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:45:51.815010 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a412b-76d3-49a0-b006-c99be1c0b541 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:51.815599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a412b-76d3-49a0-b006-c99be1c0b541 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:51.815599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a412b-76d3-49a0-b006-c99be1c0b541 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:51.816967 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 734/1476] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:45:51.822710 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7bca88-6e27-4254-a0c2-962cd3418113 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:51.823172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7bca88-6e27-4254-a0c2-962cd3418113 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:51.823289 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7bca88-6e27-4254-a0c2-962cd3418113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:51.891866 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-7b7bca88-6e27-4254-a0c2-962cd3418113 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:45:51.893080 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7b7bca88-6e27-4254-a0c2-962cd3418113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:51.991390 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 743/1477] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:45:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:45:52.016237 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a412b-76d3-49a0-b006-c99be1c0b541 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:52.017882 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f17a412b-76d3-49a0-b006-c99be1c0b541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:52.150074 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a412b-76d3-49a0-b006-c99be1c0b541 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:52.199269 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:52.246137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f95919-f5d5-41ae-b5f0-7d7a818d1f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:52.246789 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f95919-f5d5-41ae-b5f0-7d7a818d1f34 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:52.247069 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f95919-f5d5-41ae-b5f0-7d7a818d1f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:52.247389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f95919-f5d5-41ae-b5f0-7d7a818d1f34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:52.248072 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f95919-f5d5-41ae-b5f0-7d7a818d1f34 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:52.249852 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: WARNING py.warnings [None req-a1f95919-f5d5-41ae-b5f0-7d7a818d1f34 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 14:45:52.249852 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: warnings.warn(msg) Jul 26 14:45:52.249852 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: Jul 26 14:45:52.251005 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f95919-f5d5-41ae-b5f0-7d7a818d1f34 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:52.270484 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 735/1478] 10.222.0.97 () {56 vars in 1108 bytes} [Tue Jul 26 14:45:52 2022] GET /identity/v3/projects => generated 2553 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:45:54.305935 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91d222-7442-4841-a8f3-f8e523f365b7 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:54.306564 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91d222-7442-4841-a8f3-f8e523f365b7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:54.306564 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91d222-7442-4841-a8f3-f8e523f365b7 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:54.307824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 744/1479] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:45:54.312937 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb86859-b036-498f-b5c5-92d65b3198be None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:54.313337 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb86859-b036-498f-b5c5-92d65b3198be None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:54.313500 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb86859-b036-498f-b5c5-92d65b3198be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:54.379834 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-4cb86859-b036-498f-b5c5-92d65b3198be None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:45:54.381092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4cb86859-b036-498f-b5c5-92d65b3198be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:54.478685 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 736/1480] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:45:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:45:54.521371 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91d222-7442-4841-a8f3-f8e523f365b7 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:54.523520 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1a91d222-7442-4841-a8f3-f8e523f365b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:54.534388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:54.670577 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91d222-7442-4841-a8f3-f8e523f365b7 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:54.748889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:54.782466 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58fb66-a5f5-40bd-b7f0-bdb8abd83c8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:54.783097 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58fb66-a5f5-40bd-b7f0-bdb8abd83c8d service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:54.783385 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58fb66-a5f5-40bd-b7f0-bdb8abd83c8d service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:54.783689 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58fb66-a5f5-40bd-b7f0-bdb8abd83c8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:54.785315 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-cf58fb66-a5f5-40bd-b7f0-bdb8abd83c8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:54.875722 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58fb66-a5f5-40bd-b7f0-bdb8abd83c8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:54.877583 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58fb66-a5f5-40bd-b7f0-bdb8abd83c8d service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:54.895895 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 745/1481] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:45:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:45:57.106174 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b8a99-6191-4b48-96b2-bfc2fed2a750 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:57.106772 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b8a99-6191-4b48-96b2-bfc2fed2a750 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:57.106772 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b8a99-6191-4b48-96b2-bfc2fed2a750 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:57.108151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 737/1482] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:45:57.113312 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf13a85-7e30-4b60-925d-32df79f60430 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:57.113684 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf13a85-7e30-4b60-925d-32df79f60430 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:57.113870 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf13a85-7e30-4b60-925d-32df79f60430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:57.179903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0bf13a85-7e30-4b60-925d-32df79f60430 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:45:57.181108 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0bf13a85-7e30-4b60-925d-32df79f60430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:57.228917 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:57.279338 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 746/1483] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:45:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:45:57.318171 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b8a99-6191-4b48-96b2-bfc2fed2a750 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:57.319740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f77b8a99-6191-4b48-96b2-bfc2fed2a750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:57.346165 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:57.428596 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b8a99-6191-4b48-96b2-bfc2fed2a750 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:57.514512 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc39f6-c3f8-4956-9e00-b32579073dd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:57.515141 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc39f6-c3f8-4956-9e00-b32579073dd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:57.515451 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc39f6-c3f8-4956-9e00-b32579073dd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:57.515729 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc39f6-c3f8-4956-9e00-b32579073dd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:57.517377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-58dc39f6-c3f8-4956-9e00-b32579073dd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:57.596648 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:57.630442 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dc39f6-c3f8-4956-9e00-b32579073dd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:57.631911 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dc39f6-c3f8-4956-9e00-b32579073dd9 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:57.651420 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 738/1484] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:45:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:45:59.731275 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b48e0fa-ee89-4fa6-bd8d-127b499a1307 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:59.731905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b48e0fa-ee89-4fa6-bd8d-127b499a1307 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:59.731905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b48e0fa-ee89-4fa6-bd8d-127b499a1307 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:59.733213 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 747/1485] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:45:59.738331 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01918798-9d53-4b7d-a889-8b2c71015053 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:59.738750 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01918798-9d53-4b7d-a889-8b2c71015053 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:59.738880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01918798-9d53-4b7d-a889-8b2c71015053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:59.804191 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-01918798-9d53-4b7d-a889-8b2c71015053 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:45:59.805390 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-01918798-9d53-4b7d-a889-8b2c71015053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:59.900891 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 739/1486] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:45:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:45:59.938437 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b48e0fa-ee89-4fa6-bd8d-127b499a1307 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:59.939971 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1b48e0fa-ee89-4fa6-bd8d-127b499a1307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:46:00.045953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b48e0fa-ee89-4fa6-bd8d-127b499a1307 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:46:00.119341 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:46:00.121075 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b959b5-593f-42eb-a9a8-d3b9903069d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:46:00.121714 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b959b5-593f-42eb-a9a8-d3b9903069d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:46:00.121977 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b959b5-593f-42eb-a9a8-d3b9903069d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:46:00.122251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b959b5-593f-42eb-a9a8-d3b9903069d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:46:00.123897 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-03b959b5-593f-42eb-a9a8-d3b9903069d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:46:00.231770 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b959b5-593f-42eb-a9a8-d3b9903069d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:46:00.233277 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b959b5-593f-42eb-a9a8-d3b9903069d3 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:46:00.251080 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 748/1487] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:45:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:46:02.405164 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d0a31-1421-4eca-ad51-c7927c8952bf None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:46:02.405685 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d0a31-1421-4eca-ad51-c7927c8952bf None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:46:02.405794 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d0a31-1421-4eca-ad51-c7927c8952bf None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:46:02.407095 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 740/1488] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:46:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:46:02.412377 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f677f-912c-41cf-a2a8-d13e074c1d97 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:46:02.412797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f677f-912c-41cf-a2a8-d13e074c1d97 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:46:02.413012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f677f-912c-41cf-a2a8-d13e074c1d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:46:02.482479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-7b8f677f-912c-41cf-a2a8-d13e074c1d97 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:46:02.483719 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7b8f677f-912c-41cf-a2a8-d13e074c1d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:46:02.541722 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:46:02.585262 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 749/1489] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:46:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:46:02.626322 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d0a31-1421-4eca-ad51-c7927c8952bf None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:46:02.628142 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d32d0a31-1421-4eca-ad51-c7927c8952bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:46:02.664070 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:46:02.730850 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d0a31-1421-4eca-ad51-c7927c8952bf None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:46:02.805017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14874c-6784-4a12-b1af-17b5e90cc2ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:46:02.805754 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14874c-6784-4a12-b1af-17b5e90cc2ce service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:46:02.806041 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14874c-6784-4a12-b1af-17b5e90cc2ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:46:02.806318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14874c-6784-4a12-b1af-17b5e90cc2ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:46:02.808012 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0c14874c-6784-4a12-b1af-17b5e90cc2ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:46:02.888826 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:46:02.923604 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c14874c-6784-4a12-b1af-17b5e90cc2ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:46:02.924970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c14874c-6784-4a12-b1af-17b5e90cc2ce service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:46:02.944290 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 741/1490] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:46:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:46:05.450074 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f03a5-9e92-4064-b38e-e080641b1ee4 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:46:05.450631 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f03a5-9e92-4064-b38e-e080641b1ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:46:05.450631 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f03a5-9e92-4064-b38e-e080641b1ee4 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:46:05.451982 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 750/1491] 10.222.0.97 () {54 vars in 926 bytes} [Tue Jul 26 14:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:46:05.457551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50120ffd-b9d0-49a5-a499-12c901a78367 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:46:05.457988 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50120ffd-b9d0-49a5-a499-12c901a78367 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:46:05.458183 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50120ffd-b9d0-49a5-a499-12c901a78367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:46:05.530326 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-50120ffd-b9d0-49a5-a499-12c901a78367 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:46:05.531591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-50120ffd-b9d0-49a5-a499-12c901a78367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:46:05.625846 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 742/1492] 10.222.0.97 () {58 vars in 1023 bytes} [Tue Jul 26 14:46:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:46:05.666252 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f03a5-9e92-4064-b38e-e080641b1ee4 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:46:05.667960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0d7f03a5-9e92-4064-b38e-e080641b1ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:46:05.770524 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f03a5-9e92-4064-b38e-e080641b1ee4 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:46:05.842608 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d207ccdf-360b-4c49-b12e-2433f1425a20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:46:05.843219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207ccdf-360b-4c49-b12e-2433f1425a20 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:46:05.843529 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207ccdf-360b-4c49-b12e-2433f1425a20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:46:05.843806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207ccdf-360b-4c49-b12e-2433f1425a20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:46:05.845400 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d207ccdf-360b-4c49-b12e-2433f1425a20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:46:05.854347 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:46:05.953146 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d207ccdf-360b-4c49-b12e-2433f1425a20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:46:05.954533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d207ccdf-360b-4c49-b12e-2433f1425a20 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:46:05.973012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 751/1493] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:46:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:46:15.623985 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=82 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:46:15.970700 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=50 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:47:46.896580 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c321ba3-2c71-4855-9739-5d168829b827 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:47:46.897184 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c321ba3-2c71-4855-9739-5d168829b827 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:47:46.897557 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c321ba3-2c71-4855-9739-5d168829b827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:47:46.973320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3c321ba3-2c71-4855-9739-5d168829b827 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:47:46.974528 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3c321ba3-2c71-4855-9739-5d168829b827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:47:47.073858 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 743/1494] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:47:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:47:47.082726 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe090da-e1a1-4cf1-ae6d-09535dfcac94 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:47:47.083209 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe090da-e1a1-4cf1-ae6d-09535dfcac94 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:47:47.083317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe090da-e1a1-4cf1-ae6d-09535dfcac94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:47:47.159862 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-dfe090da-e1a1-4cf1-ae6d-09535dfcac94 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:47:47.160989 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-dfe090da-e1a1-4cf1-ae6d-09535dfcac94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:47:47.264419 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 752/1495] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:47:47.269288 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c321ba3-2c71-4855-9739-5d168829b827 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:47:47.270504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3c321ba3-2c71-4855-9739-5d168829b827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:47:47.282405 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c321ba3-2c71-4855-9739-5d168829b827 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:47:47.343133 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:47:47.376454 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff265c-58e1-474e-a3c5-a596603c1b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:47:47.377077 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff265c-58e1-474e-a3c5-a596603c1b6c admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:47:47.377363 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff265c-58e1-474e-a3c5-a596603c1b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:47:47.377643 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff265c-58e1-474e-a3c5-a596603c1b6c admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:47:47.378339 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff265c-58e1-474e-a3c5-a596603c1b6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:47:47.378645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff265c-58e1-474e-a3c5-a596603c1b6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:47:47.379692 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: WARNING py.warnings [None req-24ff265c-58e1-474e-a3c5-a596603c1b6c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 14:47:47.379692 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: warnings.warn(msg) Jul 26 14:47:47.379692 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: Jul 26 14:47:47.380580 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff265c-58e1-474e-a3c5-a596603c1b6c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:47:47.392130 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 744/1496] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:47:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:47:47.400456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe090da-e1a1-4cf1-ae6d-09535dfcac94 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:47:47.410290 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe090da-e1a1-4cf1-ae6d-09535dfcac94 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:47:47.471063 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:47:47.504026 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e72aa-765f-4774-a317-a176eb3b1680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:47:47.504641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e72aa-765f-4774-a317-a176eb3b1680 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:47:47.505012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e72aa-765f-4774-a317-a176eb3b1680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:47:47.505302 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e72aa-765f-4774-a317-a176eb3b1680 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:47:47.505966 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e72aa-765f-4774-a317-a176eb3b1680 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:47:47.507725 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: WARNING py.warnings [None req-898e72aa-765f-4774-a317-a176eb3b1680 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 14:47:47.507725 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: warnings.warn(msg) Jul 26 14:47:47.507725 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: Jul 26 14:47:47.508940 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e72aa-765f-4774-a317-a176eb3b1680 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:47:47.647083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 753/1497] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:47:47 2022] POST /identity/v3/projects => generated 377 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:47:47.651300 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff265c-58e1-474e-a3c5-a596603c1b6c admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:47:47.652857 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-24ff265c-58e1-474e-a3c5-a596603c1b6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:47:47.686828 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff265c-58e1-474e-a3c5-a596603c1b6c admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:47:47.780736 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b33a51-646f-4d24-9410-e88b4b15db53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:47:47.781349 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b33a51-646f-4d24-9410-e88b4b15db53 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:47:47.781630 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b33a51-646f-4d24-9410-e88b4b15db53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:47:47.781905 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b33a51-646f-4d24-9410-e88b4b15db53 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:47:47.782570 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b33a51-646f-4d24-9410-e88b4b15db53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:47:47.783604 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: WARNING py.warnings [None req-35b33a51-646f-4d24-9410-e88b4b15db53 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 14:47:47.783604 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: warnings.warn(msg) Jul 26 14:47:47.783604 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: Jul 26 14:47:47.784817 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b33a51-646f-4d24-9410-e88b4b15db53 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:47:47.915194 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 745/1498] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:47:47 2022] POST /identity/v3/users => generated 289 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:47:47.919420 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e72aa-765f-4774-a317-a176eb3b1680 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:47:47.921463 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-898e72aa-765f-4774-a317-a176eb3b1680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:47:47.983584 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e72aa-765f-4774-a317-a176eb3b1680 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:47:48.068103 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:47:48.078860 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba294e7-e033-418b-84b7-d3f2889a7780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:47:48.079571 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba294e7-e033-418b-84b7-d3f2889a7780 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:47:48.079859 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba294e7-e033-418b-84b7-d3f2889a7780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:47:48.080134 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba294e7-e033-418b-84b7-d3f2889a7780 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:47:48.080814 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba294e7-e033-418b-84b7-d3f2889a7780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:47:48.081809 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: WARNING py.warnings [None req-6ba294e7-e033-418b-84b7-d3f2889a7780 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 14:47:48.081809 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: warnings.warn(msg) Jul 26 14:47:48.081809 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: Jul 26 14:47:48.082701 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba294e7-e033-418b-84b7-d3f2889a7780 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:47:48.095667 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 754/1499] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:47:47 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:47:48.100548 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b33a51-646f-4d24-9410-e88b4b15db53 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:47:48.102123 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-35b33a51-646f-4d24-9410-e88b4b15db53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:47:48.114008 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b33a51-646f-4d24-9410-e88b4b15db53 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:47:48.138794 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:47:48.211127 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c342d1b-6960-4d7a-afb0-53f54d11048b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:47:48.211774 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c342d1b-6960-4d7a-afb0-53f54d11048b admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:47:48.212051 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c342d1b-6960-4d7a-afb0-53f54d11048b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:47:48.212324 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c342d1b-6960-4d7a-afb0-53f54d11048b admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:47:48.213012 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c342d1b-6960-4d7a-afb0-53f54d11048b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:47:48.214013 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: WARNING py.warnings [None req-5c342d1b-6960-4d7a-afb0-53f54d11048b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 14:47:48.214013 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: warnings.warn(msg) Jul 26 14:47:48.214013 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: Jul 26 14:47:48.214893 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c342d1b-6960-4d7a-afb0-53f54d11048b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:47:48.227825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 746/1500] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:47:48 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:47:48.232136 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba294e7-e033-418b-84b7-d3f2889a7780 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:47:48.233700 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6ba294e7-e033-418b-84b7-d3f2889a7780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:47:48.268183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba294e7-e033-418b-84b7-d3f2889a7780 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:47:48.361208 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5563de-bfcc-4c75-865f-3ae96090c2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:47:48.361878 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5563de-bfcc-4c75-865f-3ae96090c2c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:47:48.362164 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5563de-bfcc-4c75-865f-3ae96090c2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:47:48.362442 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5563de-bfcc-4c75-865f-3ae96090c2c1 admin admin] PATH_INFO: `/v3/projects/0b3fda84fbaa4a64bc9449dffc648135/users/6caa95094a92483abe2db50f07ccb9f7/roles/e033961a9363456a8510c34414dd839e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:47:48.385159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5563de-bfcc-4c75-865f-3ae96090c2c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b3fda84fbaa4a64bc9449dffc648135, user_id=6caa95094a92483abe2db50f07ccb9f7, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:47:48.386182 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: WARNING py.warnings [None req-9d5563de-bfcc-4c75-865f-3ae96090c2c1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 14:47:48.386182 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: warnings.warn(msg) Jul 26 14:47:48.386182 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: Jul 26 14:47:48.388176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5563de-bfcc-4c75-865f-3ae96090c2c1 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:47:48.478171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 755/1501] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:47:48 2022] PUT /identity/v3/projects/0b3fda84fbaa4a64bc9449dffc648135/users/6caa95094a92483abe2db50f07ccb9f7/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:47:48.490421 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8328f0-642b-4dc0-a2db-670952b8a18a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:47:48.490648 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8328f0-642b-4dc0-a2db-670952b8a18a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:47:48.490923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8328f0-642b-4dc0-a2db-670952b8a18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:47:48.547390 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-da8328f0-642b-4dc0-a2db-670952b8a18a None None] MFA Rules not processed for user `6caa95094a92483abe2db50f07ccb9f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:47:48.548641 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-da8328f0-642b-4dc0-a2db-670952b8a18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:47:48.625764 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 747/1502] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:47:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:47:48.639499 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946e268-100d-479c-bccc-8560171fb1e9 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:47:48.639795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946e268-100d-479c-bccc-8560171fb1e9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:47:48.640076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946e268-100d-479c-bccc-8560171fb1e9 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:47:48.641454 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 756/1503] 10.222.0.97 () {54 vars in 964 bytes} [Tue Jul 26 14:47:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:47:48.646410 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b9a39-412e-417d-ac35-c8cec28399d6 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:47:48.646694 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b9a39-412e-417d-ac35-c8cec28399d6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:47:48.647011 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b9a39-412e-417d-ac35-c8cec28399d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:47:48.678310 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:47:48.724132 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e07b9a39-412e-417d-ac35-c8cec28399d6 None None] MFA Rules not processed for user `bad45b307edc415fa6185a82af64ede4`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:47:48.725369 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e07b9a39-412e-417d-ac35-c8cec28399d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:47:48.799807 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 748/1504] 10.222.0.97 () {58 vars in 1061 bytes} [Tue Jul 26 14:47:48 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:47:48.807650 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946e268-100d-479c-bccc-8560171fb1e9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:47:48.809185 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5946e268-100d-479c-bccc-8560171fb1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:47:48.860168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:47:48.912915 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946e268-100d-479c-bccc-8560171fb1e9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:47:48.987140 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ae73f-6529-40c8-97dd-6774c79986ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:47:48.987796 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ae73f-6529-40c8-97dd-6774c79986ce service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:47:48.988082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ae73f-6529-40c8-97dd-6774c79986ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:47:48.988359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ae73f-6529-40c8-97dd-6774c79986ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:47:48.989977 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3c3ae73f-6529-40c8-97dd-6774c79986ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:47:49.088347 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ae73f-6529-40c8-97dd-6774c79986ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:47:49.089779 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ae73f-6529-40c8-97dd-6774c79986ce service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:47:49.093794 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:47:49.108555 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 757/1505] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:47:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:47:49.154382 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f118606-eba5-421b-80fe-a90f6112aa21 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:47:49.154818 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f118606-eba5-421b-80fe-a90f6112aa21 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:47:49.154972 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f118606-eba5-421b-80fe-a90f6112aa21 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:47:49.156283 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 749/1506] 10.222.0.97 () {52 vars in 830 bytes} [Tue Jul 26 14:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:47:49.159824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ae73f-6529-40c8-97dd-6774c79986ce service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:47:49.161078 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3c3ae73f-6529-40c8-97dd-6774c79986ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:47:49.263606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ae73f-6529-40c8-97dd-6774c79986ce service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:47:49.358675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e35fc62-dc1e-44a3-9d84-4550b3c296b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:47:49.359229 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e35fc62-dc1e-44a3-9d84-4550b3c296b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:47:49.359540 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e35fc62-dc1e-44a3-9d84-4550b3c296b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:47:49.359816 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e35fc62-dc1e-44a3-9d84-4550b3c296b1 admin admin] PATH_INFO: `/v3/users/6caa95094a92483abe2db50f07ccb9f7` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:47:49.372624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e35fc62-dc1e-44a3-9d84-4550b3c296b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6caa95094a92483abe2db50f07ccb9f7)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:47:49.374254 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: WARNING py.warnings [None req-3e35fc62-dc1e-44a3-9d84-4550b3c296b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 14:47:49.374254 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: warnings.warn(msg) Jul 26 14:47:49.374254 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: Jul 26 14:47:49.375495 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e35fc62-dc1e-44a3-9d84-4550b3c296b1 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:47:49.385167 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:47:49.618307 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3e35fc62-dc1e-44a3-9d84-4550b3c296b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6caa95094a92483abe2db50f07ccb9f7'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:47:49.618772 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3e35fc62-dc1e-44a3-9d84-4550b3c296b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6caa95094a92483abe2db50f07ccb9f7'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:47:49.637879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3e35fc62-dc1e-44a3-9d84-4550b3c296b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6caa95094a92483abe2db50f07ccb9f7'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:47:49.649920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3e35fc62-dc1e-44a3-9d84-4550b3c296b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6caa95094a92483abe2db50f07ccb9f7'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:47:49.650420 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3e35fc62-dc1e-44a3-9d84-4550b3c296b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6caa95094a92483abe2db50f07ccb9f7'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:47:49.757892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 758/1507] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:47:49 2022] DELETE /identity/v3/users/6caa95094a92483abe2db50f07ccb9f7 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:47:49.762452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92879893-d5da-457f-ab02-9ed940990bae None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:47:49.762680 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92879893-d5da-457f-ab02-9ed940990bae None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:47:49.762968 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92879893-d5da-457f-ab02-9ed940990bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:47:49.819732 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-92879893-d5da-457f-ab02-9ed940990bae None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:47:49.820938 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-92879893-d5da-457f-ab02-9ed940990bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:47:49.895739 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:47:49.908261 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 750/1508] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:47:49.917819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e35fc62-dc1e-44a3-9d84-4550b3c296b1 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:47:49.919096 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3e35fc62-dc1e-44a3-9d84-4550b3c296b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:47:50.020741 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e35fc62-dc1e-44a3-9d84-4550b3c296b1 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:47:50.076058 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:47:50.095816 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5753b48e-5b99-44c5-a64a-4d1a33e065a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:47:50.096447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753b48e-5b99-44c5-a64a-4d1a33e065a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:47:50.096732 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753b48e-5b99-44c5-a64a-4d1a33e065a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:47:50.097008 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753b48e-5b99-44c5-a64a-4d1a33e065a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:47:50.098638 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5753b48e-5b99-44c5-a64a-4d1a33e065a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:47:50.207517 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5753b48e-5b99-44c5-a64a-4d1a33e065a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:47:50.208879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5753b48e-5b99-44c5-a64a-4d1a33e065a9 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:47:50.230212 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 759/1509] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:47:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:47:50.796035 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92879893-d5da-457f-ab02-9ed940990bae None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:47:50.797602 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-92879893-d5da-457f-ab02-9ed940990bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:47:50.809523 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92879893-d5da-457f-ab02-9ed940990bae None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:47:50.906589 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8da763-ca52-43a8-a004-d59f6811a40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:47:50.907242 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8da763-ca52-43a8-a004-d59f6811a40e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:47:50.907568 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8da763-ca52-43a8-a004-d59f6811a40e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:47:50.907848 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8da763-ca52-43a8-a004-d59f6811a40e admin admin] PATH_INFO: `/v3/projects/0b3fda84fbaa4a64bc9449dffc648135` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:47:50.917713 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8da763-ca52-43a8-a004-d59f6811a40e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b3fda84fbaa4a64bc9449dffc648135)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:47:50.918731 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: WARNING py.warnings [None req-cf8da763-ca52-43a8-a004-d59f6811a40e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 14:47:50.918731 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: warnings.warn(msg) Jul 26 14:47:50.918731 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: Jul 26 14:47:50.920552 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8da763-ca52-43a8-a004-d59f6811a40e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:47:51.307659 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cf8da763-ca52-43a8-a004-d59f6811a40e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b3fda84fbaa4a64bc9449dffc648135', 'request_id': 'req-cf8da763-ca52-43a8-a004-d59f6811a40e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:47:51.314183 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cf8da763-ca52-43a8-a004-d59f6811a40e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b3fda84fbaa4a64bc9449dffc648135', 'request_id': 'req-cf8da763-ca52-43a8-a004-d59f6811a40e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:47:51.405693 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cf8da763-ca52-43a8-a004-d59f6811a40e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b3fda84fbaa4a64bc9449dffc648135', 'request_id': 'req-cf8da763-ca52-43a8-a004-d59f6811a40e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:47:51.439758 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cf8da763-ca52-43a8-a004-d59f6811a40e admin admin] The token cache is being invalidate because project 0b3fda84fbaa4a64bc9449dffc648135 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:47:51.440052 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cf8da763-ca52-43a8-a004-d59f6811a40e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:47:51.440593 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cf8da763-ca52-43a8-a004-d59f6811a40e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:47:51.441959 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 751/1510] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:47:50 2022] DELETE /identity/v3/projects/0b3fda84fbaa4a64bc9449dffc648135 => generated 0 bytes in 647 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:48:00.227770 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=62 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:48:01.437688 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:49:38.412061 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78789050-dfeb-4cfa-a4b7-2a707b83fb68 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:49:38.412633 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78789050-dfeb-4cfa-a4b7-2a707b83fb68 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:49:38.412633 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78789050-dfeb-4cfa-a4b7-2a707b83fb68 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:49:38.413932 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 760/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 14:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:49:38.419142 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c399e8d-aa1f-45cb-ba9a-9b112f59d009 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:49:38.419607 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c399e8d-aa1f-45cb-ba9a-9b112f59d009 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:49:38.419745 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c399e8d-aa1f-45cb-ba9a-9b112f59d009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:49:38.500375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9c399e8d-aa1f-45cb-ba9a-9b112f59d009 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:49:38.501587 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9c399e8d-aa1f-45cb-ba9a-9b112f59d009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:49:38.607296 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 752/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 14:49:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:49:38.762110 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78789050-dfeb-4cfa-a4b7-2a707b83fb68 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:49:38.764391 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-78789050-dfeb-4cfa-a4b7-2a707b83fb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:49:38.882922 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78789050-dfeb-4cfa-a4b7-2a707b83fb68 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:49:38.888480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47302018-a934-4a45-b224-0ca5558605e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:49:38.889100 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47302018-a934-4a45-b224-0ca5558605e6 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:49:38.889380 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47302018-a934-4a45-b224-0ca5558605e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:49:38.889655 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47302018-a934-4a45-b224-0ca5558605e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:49:38.898522 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47302018-a934-4a45-b224-0ca5558605e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:49:38.900133 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47302018-a934-4a45-b224-0ca5558605e6 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:49:38.909772 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 761/1513] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:49:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:49:39.737821 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a8cdf-f38a-4bad-b02e-06810302e4cd None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:49:39.738349 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a8cdf-f38a-4bad-b02e-06810302e4cd None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:49:39.738473 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a8cdf-f38a-4bad-b02e-06810302e4cd None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:49:39.740495 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 753/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 14:49:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:49:39.747644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac16e43-a7b2-49e9-b6ef-d6a1fe63f147 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:49:39.747644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac16e43-a7b2-49e9-b6ef-d6a1fe63f147 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:49:39.747644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac16e43-a7b2-49e9-b6ef-d6a1fe63f147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:49:39.766424 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-8ac16e43-a7b2-49e9-b6ef-d6a1fe63f147 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:49:39.767655 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8ac16e43-a7b2-49e9-b6ef-d6a1fe63f147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:49:39.775472 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 762/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 14:49:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:49:39.928704 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a8cdf-f38a-4bad-b02e-06810302e4cd None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:49:39.943240 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a8cdf-f38a-4bad-b02e-06810302e4cd None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:49:39.948777 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b1951-7ab4-4fb1-9b48-888e7f12b50f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:49:39.949378 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b1951-7ab4-4fb1-9b48-888e7f12b50f service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:49:39.949656 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b1951-7ab4-4fb1-9b48-888e7f12b50f service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:49:39.949980 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b1951-7ab4-4fb1-9b48-888e7f12b50f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:49:39.958082 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384b1951-7ab4-4fb1-9b48-888e7f12b50f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:49:39.959690 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384b1951-7ab4-4fb1-9b48-888e7f12b50f service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:49:39.968947 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 754/1516] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:49:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:49:40.751066 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84de54-2d9a-455e-978b-3038e594a904 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:49:40.751655 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84de54-2d9a-455e-978b-3038e594a904 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:49:40.751655 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84de54-2d9a-455e-978b-3038e594a904 None None] PATH_INFO: `/v3` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:49:40.752938 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 763/1517] 10.222.0.99 () {54 vars in 925 bytes} [Tue Jul 26 14:49:40 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:49:40.759465 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658c1d0-5fe3-4729-bb57-5a68d0b133a1 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:49:40.759921 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658c1d0-5fe3-4729-bb57-5a68d0b133a1 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:49:40.760028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658c1d0-5fe3-4729-bb57-5a68d0b133a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:49:40.818474 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3658c1d0-5fe3-4729-bb57-5a68d0b133a1 None None] MFA Rules not processed for user `252c2d2939134416a3e8efb254f03c5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:49:40.819752 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3658c1d0-5fe3-4729-bb57-5a68d0b133a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:49:40.840976 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:49:40.896926 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 755/1518] 10.222.0.99 () {58 vars in 1015 bytes} [Tue Jul 26 14:49:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:49:49.761384 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=56 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:49:50.281614 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae864e-14fe-4db3-951b-7b912a2f3006 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:49:50.281810 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae864e-14fe-4db3-951b-7b912a2f3006 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:49:50.281999 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae864e-14fe-4db3-951b-7b912a2f3006 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:49:50.283361 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 764/1519] 10.222.0.97 () {54 vars in 961 bytes} [Tue Jul 26 14:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:49:50.288501 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97b2d6-f0d6-434f-a98f-bc51aa9d2011 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:49:50.288970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97b2d6-f0d6-434f-a98f-bc51aa9d2011 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:49:50.289190 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97b2d6-f0d6-434f-a98f-bc51aa9d2011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:49:50.309413 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9e97b2d6-f0d6-434f-a98f-bc51aa9d2011 None None] MFA Rules not processed for user `252c2d2939134416a3e8efb254f03c5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:49:50.310626 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9e97b2d6-f0d6-434f-a98f-bc51aa9d2011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:49:50.320240 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 756/1520] 10.222.0.97 () {58 vars in 1058 bytes} [Tue Jul 26 14:49:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:49:50.327774 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae864e-14fe-4db3-951b-7b912a2f3006 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:49:50.347960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae864e-14fe-4db3-951b-7b912a2f3006 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:49:50.352982 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666d192c-118b-43ae-b3c5-6a953d19d015 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '252c2d2939134416a3e8efb254f03c5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:49:50.353599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d192c-118b-43ae-b3c5-6a953d19d015 service placement] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:49:50.353879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d192c-118b-43ae-b3c5-6a953d19d015 service placement] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:49:50.354152 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d192c-118b-43ae-b3c5-6a953d19d015 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:49:50.363882 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666d192c-118b-43ae-b3c5-6a953d19d015 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:49:50.365479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666d192c-118b-43ae-b3c5-6a953d19d015 service placement] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:49:50.376341 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 765/1521] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:49:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:49:51.226174 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdb34c-cdc4-428a-9200-70cf0878dba3 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:49:51.231991 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdb34c-cdc4-428a-9200-70cf0878dba3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:49:51.231991 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdb34c-cdc4-428a-9200-70cf0878dba3 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:49:51.231991 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 757/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 14:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:49:51.233269 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c459f-e41b-4ebd-812e-d4cb41aa9002 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:49:51.233702 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c459f-e41b-4ebd-812e-d4cb41aa9002 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:49:51.233859 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c459f-e41b-4ebd-812e-d4cb41aa9002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:49:51.255062 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-309c459f-e41b-4ebd-812e-d4cb41aa9002 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:49:51.256431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-309c459f-e41b-4ebd-812e-d4cb41aa9002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:49:51.264293 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 766/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 14:49:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:49:51.421409 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdb34c-cdc4-428a-9200-70cf0878dba3 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:49:51.437514 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdb34c-cdc4-428a-9200-70cf0878dba3 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:49:51.444553 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2301f3f0-482b-4254-9f98-f85ab0312730 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:49:51.449446 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301f3f0-482b-4254-9f98-f85ab0312730 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:49:51.449446 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301f3f0-482b-4254-9f98-f85ab0312730 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:49:51.449446 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301f3f0-482b-4254-9f98-f85ab0312730 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:49:51.454735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2301f3f0-482b-4254-9f98-f85ab0312730 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:49:51.456427 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2301f3f0-482b-4254-9f98-f85ab0312730 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:49:51.466090 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 758/1524] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:49:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:49:52.291222 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432bca2-21c8-4b15-a8ef-5586a57c6b6a None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:49:52.291885 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432bca2-21c8-4b15-a8ef-5586a57c6b6a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:49:52.291885 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432bca2-21c8-4b15-a8ef-5586a57c6b6a None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:49:52.293175 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 767/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 14:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:49:52.298277 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef626cdb-7cf0-47bf-966c-67c5a66cf585 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:49:52.298565 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef626cdb-7cf0-47bf-966c-67c5a66cf585 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:49:52.298840 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef626cdb-7cf0-47bf-966c-67c5a66cf585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:49:52.319313 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ef626cdb-7cf0-47bf-966c-67c5a66cf585 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:49:52.320539 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ef626cdb-7cf0-47bf-966c-67c5a66cf585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:49:52.328412 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 759/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 14:49:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:49:52.481586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432bca2-21c8-4b15-a8ef-5586a57c6b6a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:49:52.495934 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432bca2-21c8-4b15-a8ef-5586a57c6b6a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:49:52.501343 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf861ea-a41c-4519-9194-192bf7d47905 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:49:52.501943 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf861ea-a41c-4519-9194-192bf7d47905 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:49:52.502223 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf861ea-a41c-4519-9194-192bf7d47905 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:49:52.502496 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf861ea-a41c-4519-9194-192bf7d47905 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:49:52.510556 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf861ea-a41c-4519-9194-192bf7d47905 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:49:52.512219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf861ea-a41c-4519-9194-192bf7d47905 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:49:52.522064 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 768/1527] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:49:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:49:56.733585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54df597-29c0-4324-8420-e375be221ea8 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:49:56.734121 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54df597-29c0-4324-8420-e375be221ea8 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:49:56.734121 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54df597-29c0-4324-8420-e375be221ea8 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:49:56.735428 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 760/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 14:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:49:56.741437 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff3c5a-c033-4531-91b7-0f24f6f3a84d None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:49:56.741868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff3c5a-c033-4531-91b7-0f24f6f3a84d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:49:56.742036 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff3c5a-c033-4531-91b7-0f24f6f3a84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:49:56.761116 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-bcff3c5a-c033-4531-91b7-0f24f6f3a84d None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:49:56.762338 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-bcff3c5a-c033-4531-91b7-0f24f6f3a84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:49:56.770462 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 769/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 14:49:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:49:56.845532 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54df597-29c0-4324-8420-e375be221ea8 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:49:56.847888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b54df597-29c0-4324-8420-e375be221ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:49:56.928898 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54df597-29c0-4324-8420-e375be221ea8 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:49:56.932730 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50e997-fd97-4b01-a8e7-0a3529a5f514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:49:56.933352 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50e997-fd97-4b01-a8e7-0a3529a5f514 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:49:56.933637 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50e997-fd97-4b01-a8e7-0a3529a5f514 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:49:56.933915 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50e997-fd97-4b01-a8e7-0a3529a5f514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:49:56.943506 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50e997-fd97-4b01-a8e7-0a3529a5f514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:49:56.944985 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50e997-fd97-4b01-a8e7-0a3529a5f514 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:49:56.954416 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 761/1530] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:49:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:49:58.973095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd40c7-1c14-4d17-9b36-bc290e5bdcdf None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:49:58.973609 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd40c7-1c14-4d17-9b36-bc290e5bdcdf None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:49:58.973731 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd40c7-1c14-4d17-9b36-bc290e5bdcdf None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:49:58.974996 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 770/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 14:49:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:49:58.980317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99982d-e507-4e71-8805-ba889e342137 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:49:58.980736 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99982d-e507-4e71-8805-ba889e342137 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:49:58.980863 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99982d-e507-4e71-8805-ba889e342137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:49:59.000634 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-2c99982d-e507-4e71-8805-ba889e342137 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:49:59.001822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2c99982d-e507-4e71-8805-ba889e342137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:49:59.009526 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 762/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 14:49:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:49:59.044895 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd40c7-1c14-4d17-9b36-bc290e5bdcdf None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:49:59.056889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd40c7-1c14-4d17-9b36-bc290e5bdcdf None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:49:59.060355 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0d38c-c577-438e-9031-4f77e35f0f7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:49:59.061010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0d38c-c577-438e-9031-4f77e35f0f7d service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:49:59.061290 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0d38c-c577-438e-9031-4f77e35f0f7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:49:59.061585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0d38c-c577-438e-9031-4f77e35f0f7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:49:59.070295 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0d38c-c577-438e-9031-4f77e35f0f7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:49:59.071781 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0d38c-c577-438e-9031-4f77e35f0f7d service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:49:59.081084 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 771/1533] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:49:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:50:02.572905 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb50e0-d26b-41b0-94e2-617b32864bfd None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:50:02.573480 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb50e0-d26b-41b0-94e2-617b32864bfd None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:50:02.573480 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb50e0-d26b-41b0-94e2-617b32864bfd None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:50:02.574815 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 763/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 14:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:50:02.579952 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cf95e-f6eb-45b0-aa6d-95a77508191e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:50:02.580359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cf95e-f6eb-45b0-aa6d-95a77508191e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:50:02.580511 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cf95e-f6eb-45b0-aa6d-95a77508191e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:50:02.599987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-264cf95e-f6eb-45b0-aa6d-95a77508191e None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:50:02.601176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-264cf95e-f6eb-45b0-aa6d-95a77508191e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:50:02.608908 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 772/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 14:50:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:50:02.912745 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb50e0-d26b-41b0-94e2-617b32864bfd None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:50:02.914457 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b8cb50e0-d26b-41b0-94e2-617b32864bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:50:02.980995 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:50:02.994467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb50e0-d26b-41b0-94e2-617b32864bfd None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:50:02.998141 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e742d2b-9df7-41ef-b9bf-dc7d4b78be98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:50:03.001054 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e742d2b-9df7-41ef-b9bf-dc7d4b78be98 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:50:03.001054 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e742d2b-9df7-41ef-b9bf-dc7d4b78be98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:50:03.001054 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e742d2b-9df7-41ef-b9bf-dc7d4b78be98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:50:03.008394 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e742d2b-9df7-41ef-b9bf-dc7d4b78be98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:50:03.009827 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e742d2b-9df7-41ef-b9bf-dc7d4b78be98 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:50:03.020390 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 764/1536] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:50:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:50:05.070191 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b96a1-b9bb-49dc-b34e-f8b6ecdcaea5 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:50:05.070754 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b96a1-b9bb-49dc-b34e-f8b6ecdcaea5 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:50:05.070754 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b96a1-b9bb-49dc-b34e-f8b6ecdcaea5 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:50:05.072324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 773/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 14:50:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:50:05.077614 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f08b99-93b7-4e98-a379-6aabf2b09898 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:50:05.078090 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f08b99-93b7-4e98-a379-6aabf2b09898 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:50:05.078229 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f08b99-93b7-4e98-a379-6aabf2b09898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:50:05.098770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-02f08b99-93b7-4e98-a379-6aabf2b09898 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:50:05.100050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-02f08b99-93b7-4e98-a379-6aabf2b09898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:50:05.110087 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 765/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 14:50:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:50:05.420507 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b96a1-b9bb-49dc-b34e-f8b6ecdcaea5 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:50:05.422178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-254b96a1-b9bb-49dc-b34e-f8b6ecdcaea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:50:05.434922 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b96a1-b9bb-49dc-b34e-f8b6ecdcaea5 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:50:05.443379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51f36f9-514e-461a-ac40-b273f3245cea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:50:05.443379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f36f9-514e-461a-ac40-b273f3245cea service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:50:05.443379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f36f9-514e-461a-ac40-b273f3245cea service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:50:05.443379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f36f9-514e-461a-ac40-b273f3245cea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:50:05.449247 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51f36f9-514e-461a-ac40-b273f3245cea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:50:05.450774 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51f36f9-514e-461a-ac40-b273f3245cea service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:50:05.460618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 774/1539] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:50:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:50:15.093188 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=11 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:50:15.456035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=33 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:51:39.292923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6b891-fb49-40cb-b7d5-b9b95a082756 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:39.293923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6b891-fb49-40cb-b7d5-b9b95a082756 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:39.294376 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6b891-fb49-40cb-b7d5-b9b95a082756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:39.321404 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a2e6b891-fb49-40cb-b7d5-b9b95a082756 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:39.322924 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a2e6b891-fb49-40cb-b7d5-b9b95a082756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:39.335446 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 766/1540] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:39.345492 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba3f56-d418-4038-bbf1-f443bdfed9cd None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:39.346092 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba3f56-d418-4038-bbf1-f443bdfed9cd None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:39.346092 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba3f56-d418-4038-bbf1-f443bdfed9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:39.371483 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-37ba3f56-d418-4038-bbf1-f443bdfed9cd None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:39.373012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-37ba3f56-d418-4038-bbf1-f443bdfed9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:39.386323 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 775/1541] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:39.391035 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6b891-fb49-40cb-b7d5-b9b95a082756 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:39.407046 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6b891-fb49-40cb-b7d5-b9b95a082756 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:39.416382 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd046297-b393-4156-b9fa-21aaa79fc280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:39.417253 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd046297-b393-4156-b9fa-21aaa79fc280 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:39.417703 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd046297-b393-4156-b9fa-21aaa79fc280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:39.418145 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd046297-b393-4156-b9fa-21aaa79fc280 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:39.419022 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd046297-b393-4156-b9fa-21aaa79fc280 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:51:39.419538 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd046297-b393-4156-b9fa-21aaa79fc280 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:39.421300 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd046297-b393-4156-b9fa-21aaa79fc280 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:39.441036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 767/1542] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:51:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:51:39.449927 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba3f56-d418-4038-bbf1-f443bdfed9cd None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:39.466355 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba3f56-d418-4038-bbf1-f443bdfed9cd None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:39.471422 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0d9c8-abfa-403d-9a8d-ee425af4815d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:39.472055 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0d9c8-abfa-403d-9a8d-ee425af4815d admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:39.472336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0d9c8-abfa-403d-9a8d-ee425af4815d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:39.472612 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0d9c8-abfa-403d-9a8d-ee425af4815d admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:39.473278 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e0d9c8-abfa-403d-9a8d-ee425af4815d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:39.475184 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e0d9c8-abfa-403d-9a8d-ee425af4815d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:39.486045 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a293fc-96b4-4cc2-80f6-c4215d207f06 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:39.486563 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a293fc-96b4-4cc2-80f6-c4215d207f06 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:39.487031 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a293fc-96b4-4cc2-80f6-c4215d207f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:39.519089 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c4a293fc-96b4-4cc2-80f6-c4215d207f06 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:39.520682 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c4a293fc-96b4-4cc2-80f6-c4215d207f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:39.535060 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 768/1543] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:39.544351 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d10cc-c3a2-4a3d-a2e5-5571edabab22 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:39.544940 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d10cc-c3a2-4a3d-a2e5-5571edabab22 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:39.545419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d10cc-c3a2-4a3d-a2e5-5571edabab22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:39.573568 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-297d10cc-c3a2-4a3d-a2e5-5571edabab22 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:39.575205 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-297d10cc-c3a2-4a3d-a2e5-5571edabab22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:39.590332 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 769/1544] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:51:39.592107 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae4cc3-0812-485e-92b2-1f4c11ba7ff3 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:39.592247 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 776/1545] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:51:39 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:51:39.592792 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae4cc3-0812-485e-92b2-1f4c11ba7ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:39.593258 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0d9c8-abfa-403d-9a8d-ee425af4815d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:39.593469 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae4cc3-0812-485e-92b2-1f4c11ba7ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:39.624520 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a0ae4cc3-0812-485e-92b2-1f4c11ba7ff3 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:39.625734 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a0ae4cc3-0812-485e-92b2-1f4c11ba7ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:39.680141 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0d9c8-abfa-403d-9a8d-ee425af4815d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:39.692722 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a70f22-ab66-4117-8210-e4ed78afedf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:39.693358 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a70f22-ab66-4117-8210-e4ed78afedf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:39.693640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a70f22-ab66-4117-8210-e4ed78afedf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:39.693911 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a70f22-ab66-4117-8210-e4ed78afedf5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:39.694607 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a70f22-ab66-4117-8210-e4ed78afedf5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:51:39.695040 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a70f22-ab66-4117-8210-e4ed78afedf5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:39.696093 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: WARNING py.warnings [None req-f3a70f22-ab66-4117-8210-e4ed78afedf5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 14:51:39.696093 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: warnings.warn(msg) Jul 26 14:51:39.696093 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: Jul 26 14:51:39.696987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a70f22-ab66-4117-8210-e4ed78afedf5 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:39.716226 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 777/1546] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:51:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:51:39.720182 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e49ca-d8d9-4c88-8a9a-30ea1915ded4 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:39.720182 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e49ca-d8d9-4c88-8a9a-30ea1915ded4 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:39.720182 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e49ca-d8d9-4c88-8a9a-30ea1915ded4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:39.743165 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-7a9e49ca-d8d9-4c88-8a9a-30ea1915ded4 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:39.744413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7a9e49ca-d8d9-4c88-8a9a-30ea1915ded4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:39.745952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 770/1547] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:39.747334 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367c309-b74e-40cc-ab72-f9b2ece37641 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:39.747661 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367c309-b74e-40cc-ab72-f9b2ece37641 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:39.747938 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367c309-b74e-40cc-ab72-f9b2ece37641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:39.752925 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 778/1548] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:39.753567 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e49ca-d8d9-4c88-8a9a-30ea1915ded4 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:39.764058 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e49ca-d8d9-4c88-8a9a-30ea1915ded4 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:39.769417 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a367c309-b74e-40cc-ab72-f9b2ece37641 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:39.769650 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2736b-f89a-4814-8cb2-56d911132e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:39.770258 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2736b-f89a-4814-8cb2-56d911132e7f admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:39.770608 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2736b-f89a-4814-8cb2-56d911132e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:39.770720 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a367c309-b74e-40cc-ab72-f9b2ece37641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:39.770885 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2736b-f89a-4814-8cb2-56d911132e7f admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:39.771579 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2736b-f89a-4814-8cb2-56d911132e7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:39.772581 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: WARNING py.warnings [None req-bdc2736b-f89a-4814-8cb2-56d911132e7f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 14:51:39.772581 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: warnings.warn(msg) Jul 26 14:51:39.772581 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: Jul 26 14:51:39.773780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2736b-f89a-4814-8cb2-56d911132e7f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:39.778553 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 771/1549] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:39.779773 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41943fa7-bd27-4dc4-a8f0-428c7b6b6f32 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:39.780067 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41943fa7-bd27-4dc4-a8f0-428c7b6b6f32 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:39.780350 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41943fa7-bd27-4dc4-a8f0-428c7b6b6f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:39.799771 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-41943fa7-bd27-4dc4-a8f0-428c7b6b6f32 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:39.800896 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-41943fa7-bd27-4dc4-a8f0-428c7b6b6f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:39.808666 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 772/1550] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:39.809830 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4bbc0-6908-4b86-bdfc-084364f3aec7 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:39.810121 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4bbc0-6908-4b86-bdfc-084364f3aec7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:39.810408 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4bbc0-6908-4b86-bdfc-084364f3aec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:39.828869 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-d9e4bbc0-6908-4b86-bdfc-084364f3aec7 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:39.830586 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d9e4bbc0-6908-4b86-bdfc-084364f3aec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:39.838500 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 773/1551] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:39.839810 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4bbc0-6908-4b86-bdfc-084364f3aec7 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:39.848952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4bbc0-6908-4b86-bdfc-084364f3aec7 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:39.853513 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf7732-e179-4311-9b57-374894af93ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:39.854102 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf7732-e179-4311-9b57-374894af93ca admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:39.854388 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf7732-e179-4311-9b57-374894af93ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:39.854667 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf7732-e179-4311-9b57-374894af93ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:39.855317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf7732-e179-4311-9b57-374894af93ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:39.856367 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: WARNING py.warnings [None req-94cf7732-e179-4311-9b57-374894af93ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 14:51:39.856367 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: warnings.warn(msg) Jul 26 14:51:39.856367 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: Jul 26 14:51:39.857568 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf7732-e179-4311-9b57-374894af93ca admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:39.871632 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 779/1552] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:51:39 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:51:39.874776 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592083a3-9190-423e-8170-c5c4f5dd4924 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:39.874776 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592083a3-9190-423e-8170-c5c4f5dd4924 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:39.874776 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592083a3-9190-423e-8170-c5c4f5dd4924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:39.893623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-592083a3-9190-423e-8170-c5c4f5dd4924 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:39.894832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-592083a3-9190-423e-8170-c5c4f5dd4924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:39.903021 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 780/1553] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:39.906409 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439fc49-40e9-45bd-bb4d-93c1c9588bb3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:39.906409 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439fc49-40e9-45bd-bb4d-93c1c9588bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:39.906409 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439fc49-40e9-45bd-bb4d-93c1c9588bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:39.925333 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-b439fc49-40e9-45bd-bb4d-93c1c9588bb3 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:39.926625 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b439fc49-40e9-45bd-bb4d-93c1c9588bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:39.934628 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 781/1554] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:39.938348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2206e6-899d-4d9e-ae21-5c53960afab5 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:39.938348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2206e6-899d-4d9e-ae21-5c53960afab5 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:39.938348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2206e6-899d-4d9e-ae21-5c53960afab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:39.952019 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 774/1555] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:51:39 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:51:39.954541 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3887dc-0ba0-424d-8fd8-ea83d5d62225 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:39.954541 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3887dc-0ba0-424d-8fd8-ea83d5d62225 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:39.954541 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3887dc-0ba0-424d-8fd8-ea83d5d62225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:39.960818 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ae2206e6-899d-4d9e-ae21-5c53960afab5 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:39.962076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ae2206e6-899d-4d9e-ae21-5c53960afab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:39.973954 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-bc3887dc-0ba0-424d-8fd8-ea83d5d62225 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:39.975296 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-bc3887dc-0ba0-424d-8fd8-ea83d5d62225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:40.022027 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 782/1556] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:40.023305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff12a0-5531-4012-8ee0-13dd5dd06d19 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:40.023629 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff12a0-5531-4012-8ee0-13dd5dd06d19 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:40.023912 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff12a0-5531-4012-8ee0-13dd5dd06d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:40.043614 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-9cff12a0-5531-4012-8ee0-13dd5dd06d19 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:40.044352 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 775/1557] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:40.044828 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9cff12a0-5531-4012-8ee0-13dd5dd06d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:40.045697 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600cba7-5bf6-4e49-8022-acda646c9a8f None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:40.045991 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600cba7-5bf6-4e49-8022-acda646c9a8f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:40.046268 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600cba7-5bf6-4e49-8022-acda646c9a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:40.057026 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 783/1558] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:40.057815 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff12a0-5531-4012-8ee0-13dd5dd06d19 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:40.067561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff12a0-5531-4012-8ee0-13dd5dd06d19 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:40.070220 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b600cba7-5bf6-4e49-8022-acda646c9a8f None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:40.071551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b600cba7-5bf6-4e49-8022-acda646c9a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:40.072184 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22cd86c-01f3-44f8-9a17-ca9cec1501af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:40.072862 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22cd86c-01f3-44f8-9a17-ca9cec1501af admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:40.073140 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22cd86c-01f3-44f8-9a17-ca9cec1501af admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:40.073412 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22cd86c-01f3-44f8-9a17-ca9cec1501af admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:40.074102 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22cd86c-01f3-44f8-9a17-ca9cec1501af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:40.075719 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22cd86c-01f3-44f8-9a17-ca9cec1501af admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:40.079760 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 776/1559] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:40.081042 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafebde2-591a-461b-8b42-930f05151eaf None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:40.081347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafebde2-591a-461b-8b42-930f05151eaf None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:40.081624 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafebde2-591a-461b-8b42-930f05151eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:40.088622 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 784/1560] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:51:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:51:40.088622 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22cd86c-01f3-44f8-9a17-ca9cec1501af admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:40.093067 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:51:40.098592 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22cd86c-01f3-44f8-9a17-ca9cec1501af admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:40.102763 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-bafebde2-591a-461b-8b42-930f05151eaf None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:40.107985 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3f222-1015-460d-bd8d-3645d2556139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:40.107985 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3f222-1015-460d-bd8d-3645d2556139 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:40.108212 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-bafebde2-591a-461b-8b42-930f05151eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:40.108310 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3f222-1015-460d-bd8d-3645d2556139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:40.108310 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3f222-1015-460d-bd8d-3645d2556139 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:40.108310 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c3f222-1015-460d-bd8d-3645d2556139 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:51:40.108310 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c3f222-1015-460d-bd8d-3645d2556139 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:40.108310 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c3f222-1015-460d-bd8d-3645d2556139 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:40.112734 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 777/1561] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:40.113558 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafebde2-591a-461b-8b42-930f05151eaf None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:40.127767 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 785/1562] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:51:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:51:40.127767 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3f222-1015-460d-bd8d-3645d2556139 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:40.128053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafebde2-591a-461b-8b42-930f05151eaf None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:40.128351 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c074e08-be62-42e5-a584-ce054ceae2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:40.128964 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c074e08-be62-42e5-a584-ce054ceae2d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:40.129249 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c074e08-be62-42e5-a584-ce054ceae2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:40.129521 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c074e08-be62-42e5-a584-ce054ceae2d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:40.130176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c074e08-be62-42e5-a584-ce054ceae2d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:40.132178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c074e08-be62-42e5-a584-ce054ceae2d9 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:40.137807 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3f222-1015-460d-bd8d-3645d2556139 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:40.142758 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e574f6-bdbe-4c1c-8247-60c884dcedea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:40.143501 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e574f6-bdbe-4c1c-8247-60c884dcedea admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:40.143795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e574f6-bdbe-4c1c-8247-60c884dcedea admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:40.144125 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:51:40.144263 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e574f6-bdbe-4c1c-8247-60c884dcedea admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:40.144987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e574f6-bdbe-4c1c-8247-60c884dcedea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:51:40.145359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e574f6-bdbe-4c1c-8247-60c884dcedea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:40.146980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e574f6-bdbe-4c1c-8247-60c884dcedea admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:40.159088 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 786/1563] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:51:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:51:40.159905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e574f6-bdbe-4c1c-8247-60c884dcedea admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:40.169518 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e574f6-bdbe-4c1c-8247-60c884dcedea admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:40.174575 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d7257-1c59-416c-a24c-d24a9e1d8da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:40.175191 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d7257-1c59-416c-a24c-d24a9e1d8da2 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:40.175505 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d7257-1c59-416c-a24c-d24a9e1d8da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:40.175782 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d7257-1c59-416c-a24c-d24a9e1d8da2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:40.176456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d7257-1c59-416c-a24c-d24a9e1d8da2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:51:40.176801 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d7257-1c59-416c-a24c-d24a9e1d8da2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:40.178419 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d7257-1c59-416c-a24c-d24a9e1d8da2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:40.190934 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 787/1564] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:51:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:51:40.191853 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d7257-1c59-416c-a24c-d24a9e1d8da2 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:40.201000 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d7257-1c59-416c-a24c-d24a9e1d8da2 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:40.206081 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7e476c-2bec-4a31-9bbf-85e4a24b5527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:40.206685 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e476c-2bec-4a31-9bbf-85e4a24b5527 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:40.206985 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e476c-2bec-4a31-9bbf-85e4a24b5527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:40.207262 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e476c-2bec-4a31-9bbf-85e4a24b5527 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:40.207984 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7e476c-2bec-4a31-9bbf-85e4a24b5527 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:51:40.208290 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7e476c-2bec-4a31-9bbf-85e4a24b5527 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:40.209822 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7e476c-2bec-4a31-9bbf-85e4a24b5527 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:40.221280 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 788/1565] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:51:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:51:40.222232 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7e476c-2bec-4a31-9bbf-85e4a24b5527 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:40.231834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7e476c-2bec-4a31-9bbf-85e4a24b5527 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:40.236455 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b14a2-51f8-4f43-acfc-e4a305b510ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:40.237050 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b14a2-51f8-4f43-acfc-e4a305b510ff admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:40.237338 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b14a2-51f8-4f43-acfc-e4a305b510ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:40.237616 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b14a2-51f8-4f43-acfc-e4a305b510ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:40.238296 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b14a2-51f8-4f43-acfc-e4a305b510ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:51:40.238605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b14a2-51f8-4f43-acfc-e4a305b510ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:40.240224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b14a2-51f8-4f43-acfc-e4a305b510ff admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:40.252709 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 789/1566] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:51:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:51:40.253628 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b14a2-51f8-4f43-acfc-e4a305b510ff admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:40.262748 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b14a2-51f8-4f43-acfc-e4a305b510ff admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:40.268018 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed31ad7f-129a-450a-9778-95b770c37ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:40.268591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31ad7f-129a-450a-9778-95b770c37ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:40.268879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31ad7f-129a-450a-9778-95b770c37ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:40.269162 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31ad7f-129a-450a-9778-95b770c37ef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:40.269829 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed31ad7f-129a-450a-9778-95b770c37ef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:40.271367 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed31ad7f-129a-450a-9778-95b770c37ef4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:40.281083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 790/1567] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:51:40 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:51:40.282001 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed31ad7f-129a-450a-9778-95b770c37ef4 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:40.288172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 778/1568] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:51:40 2022] POST /identity/v3/users => generated 290 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:51:40.288172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c074e08-be62-42e5-a584-ce054ceae2d9 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:40.293002 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed31ad7f-129a-450a-9778-95b770c37ef4 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:40.297605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f0807-35ac-410b-8a5f-c39add578989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:40.298209 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f0807-35ac-410b-8a5f-c39add578989 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:40.298497 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f0807-35ac-410b-8a5f-c39add578989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:40.298777 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f0807-35ac-410b-8a5f-c39add578989 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:40.298963 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c074e08-be62-42e5-a584-ce054ceae2d9 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:40.299501 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f0807-35ac-410b-8a5f-c39add578989 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:51:40.299813 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f0807-35ac-410b-8a5f-c39add578989 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:40.301404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f0807-35ac-410b-8a5f-c39add578989 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:40.303708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e55817-fdfc-402f-8e21-438e25faf1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:40.304382 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e55817-fdfc-402f-8e21-438e25faf1bb admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:40.304613 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e55817-fdfc-402f-8e21-438e25faf1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:40.304894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e55817-fdfc-402f-8e21-438e25faf1bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:40.305556 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e55817-fdfc-402f-8e21-438e25faf1bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:40.307496 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e55817-fdfc-402f-8e21-438e25faf1bb admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:40.314447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 791/1569] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:51:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:51:40.315225 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f0807-35ac-410b-8a5f-c39add578989 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:40.324448 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f0807-35ac-410b-8a5f-c39add578989 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:40.328981 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a17437-7147-4164-ba67-32b1f4742b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:40.330172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a17437-7147-4164-ba67-32b1f4742b61 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:40.330455 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a17437-7147-4164-ba67-32b1f4742b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:40.330725 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a17437-7147-4164-ba67-32b1f4742b61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:40.331375 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a17437-7147-4164-ba67-32b1f4742b61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:40.333298 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a17437-7147-4164-ba67-32b1f4742b61 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:40.461719 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 779/1570] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:51:40 2022] POST /identity/v3/projects => generated 393 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:51:40.462547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e55817-fdfc-402f-8e21-438e25faf1bb admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:40.481208 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 792/1571] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:51:40 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:51:40.481208 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a17437-7147-4164-ba67-32b1f4742b61 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:40.531003 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e55817-fdfc-402f-8e21-438e25faf1bb admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:40.547424 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a17437-7147-4164-ba67-32b1f4742b61 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:40.552988 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780ffea-5848-41ef-aaef-a8d659e787c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:40.553773 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780ffea-5848-41ef-aaef-a8d659e787c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:40.554082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780ffea-5848-41ef-aaef-a8d659e787c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:40.554358 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780ffea-5848-41ef-aaef-a8d659e787c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:40.555152 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780ffea-5848-41ef-aaef-a8d659e787c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:40.557111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780ffea-5848-41ef-aaef-a8d659e787c3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:40.586662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e28ad-1d56-4899-81e5-0290b094da57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:40.587291 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e28ad-1d56-4899-81e5-0290b094da57 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:40.587606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e28ad-1d56-4899-81e5-0290b094da57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:40.587886 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e28ad-1d56-4899-81e5-0290b094da57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:40.588559 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32e28ad-1d56-4899-81e5-0290b094da57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:40.590496 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32e28ad-1d56-4899-81e5-0290b094da57 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:40.787592 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 793/1572] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:51:40 2022] POST /identity/v3/projects => generated 375 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:51:40.788556 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780ffea-5848-41ef-aaef-a8d659e787c3 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:40.799775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 780/1573] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:51:40 2022] POST /identity/v3/projects => generated 371 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:51:40.800606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e28ad-1d56-4899-81e5-0290b094da57 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:40.829461 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:51:40.852247 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780ffea-5848-41ef-aaef-a8d659e787c3 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:40.865486 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e28ad-1d56-4899-81e5-0290b094da57 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:40.870762 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40ead74-5a87-4a5d-b9d3-010521b7b65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:40.871469 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ead74-5a87-4a5d-b9d3-010521b7b65d admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:40.871760 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ead74-5a87-4a5d-b9d3-010521b7b65d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:40.872041 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ead74-5a87-4a5d-b9d3-010521b7b65d admin admin] PATH_INFO: `/v3/projects/ace4455a9e1b44e5bf7b04fd4abf81d7/users/de3b593763e44e28be0f51187defa75d/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:40.883494 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:51:40.887518 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40ead74-5a87-4a5d-b9d3-010521b7b65d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ace4455a9e1b44e5bf7b04fd4abf81d7, user_id=de3b593763e44e28be0f51187defa75d, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:40.892818 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: WARNING py.warnings [None req-a40ead74-5a87-4a5d-b9d3-010521b7b65d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 14:51:40.892818 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: warnings.warn(msg) Jul 26 14:51:40.892818 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: Jul 26 14:51:40.892818 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40ead74-5a87-4a5d-b9d3-010521b7b65d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:40.909851 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72242a-db53-45a1-b30e-da527ea375bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:40.910476 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72242a-db53-45a1-b30e-da527ea375bd admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:40.910766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72242a-db53-45a1-b30e-da527ea375bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:40.911044 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72242a-db53-45a1-b30e-da527ea375bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:40.911771 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72242a-db53-45a1-b30e-da527ea375bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:40.913738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72242a-db53-45a1-b30e-da527ea375bd admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:41.002935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 781/1574] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:51:40 2022] PUT /identity/v3/projects/ace4455a9e1b44e5bf7b04fd4abf81d7/users/de3b593763e44e28be0f51187defa75d/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:51:41.006435 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40ead74-5a87-4a5d-b9d3-010521b7b65d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:41.052849 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 794/1575] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:51:40 2022] POST /identity/v3/projects => generated 373 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:51:41.053660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72242a-db53-45a1-b30e-da527ea375bd admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:41.072253 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40ead74-5a87-4a5d-b9d3-010521b7b65d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:41.121097 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72242a-db53-45a1-b30e-da527ea375bd admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:41.126742 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1339c3-28f9-41b9-bb88-536a88089a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:41.127366 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1339c3-28f9-41b9-bb88-536a88089a8e admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:41.127691 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1339c3-28f9-41b9-bb88-536a88089a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:41.127969 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1339c3-28f9-41b9-bb88-536a88089a8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:41.128638 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1339c3-28f9-41b9-bb88-536a88089a8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:41.130562 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1339c3-28f9-41b9-bb88-536a88089a8e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:41.131368 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabbc35-02d4-401c-a21f-f7b8f22723f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:41.132026 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabbc35-02d4-401c-a21f-f7b8f22723f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:41.132350 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabbc35-02d4-401c-a21f-f7b8f22723f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:41.132645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabbc35-02d4-401c-a21f-f7b8f22723f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:41.133548 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fabbc35-02d4-401c-a21f-f7b8f22723f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:41.135389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fabbc35-02d4-401c-a21f-f7b8f22723f7 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:41.147036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 782/1576] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:51:41.147875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabbc35-02d4-401c-a21f-f7b8f22723f7 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:41.157410 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabbc35-02d4-401c-a21f-f7b8f22723f7 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:41.162710 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d4e7ce-0f40-48f9-9dd8-a5b3f77acdbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:41.163300 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d4e7ce-0f40-48f9-9dd8-a5b3f77acdbe admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:41.163618 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d4e7ce-0f40-48f9-9dd8-a5b3f77acdbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:41.163896 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d4e7ce-0f40-48f9-9dd8-a5b3f77acdbe admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:41.164544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d4e7ce-0f40-48f9-9dd8-a5b3f77acdbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:41.166452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d4e7ce-0f40-48f9-9dd8-a5b3f77acdbe admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:41.239206 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 795/1577] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:51:41 2022] POST /identity/v3/projects => generated 373 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:51:41.239206 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1339c3-28f9-41b9-bb88-536a88089a8e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:41.263319 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 783/1578] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:51:41 2022] POST /identity/v3/users => generated 297 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:51:41.263319 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d4e7ce-0f40-48f9-9dd8-a5b3f77acdbe admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:41.272661 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d4e7ce-0f40-48f9-9dd8-a5b3f77acdbe admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:41.283789 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:51:41.294034 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1339c3-28f9-41b9-bb88-536a88089a8e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:41.299084 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd088a1-dfd7-439a-ac30-a6a59a41c94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:41.299741 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd088a1-dfd7-439a-ac30-a6a59a41c94d admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:41.300026 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd088a1-dfd7-439a-ac30-a6a59a41c94d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:41.300335 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd088a1-dfd7-439a-ac30-a6a59a41c94d admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:41.301013 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd088a1-dfd7-439a-ac30-a6a59a41c94d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:41.302960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd088a1-dfd7-439a-ac30-a6a59a41c94d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:41.334588 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01907c6a-602f-43ae-ae9f-5065e4be9a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:41.335218 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01907c6a-602f-43ae-ae9f-5065e4be9a21 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:41.335538 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01907c6a-602f-43ae-ae9f-5065e4be9a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:41.335818 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01907c6a-602f-43ae-ae9f-5065e4be9a21 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:41.336480 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01907c6a-602f-43ae-ae9f-5065e4be9a21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:41.338483 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01907c6a-602f-43ae-ae9f-5065e4be9a21 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:41.391773 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 796/1579] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:51:41 2022] POST /identity/v3/users => generated 290 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:51:41.392514 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd088a1-dfd7-439a-ac30-a6a59a41c94d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:41.401704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd088a1-dfd7-439a-ac30-a6a59a41c94d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:41.406282 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5bf5f-acda-40dd-8f55-d047f6f50901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:41.406872 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5bf5f-acda-40dd-8f55-d047f6f50901 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:41.407159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5bf5f-acda-40dd-8f55-d047f6f50901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:41.407462 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5bf5f-acda-40dd-8f55-d047f6f50901 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:41.408109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a5bf5f-acda-40dd-8f55-d047f6f50901 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:41.410020 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a5bf5f-acda-40dd-8f55-d047f6f50901 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:41.472690 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 784/1580] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:51:41 2022] POST /identity/v3/users => generated 288 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:51:41.473972 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a9f7c-8ea9-48c4-b676-9ab010cb2616 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:41.474268 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a9f7c-8ea9-48c4-b676-9ab010cb2616 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:41.474550 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a9f7c-8ea9-48c4-b676-9ab010cb2616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:41.482304 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:51:41.493721 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3a0a9f7c-8ea9-48c4-b676-9ab010cb2616 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:41.494927 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3a0a9f7c-8ea9-48c4-b676-9ab010cb2616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:41.503092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 785/1581] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:41.503870 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a9f7c-8ea9-48c4-b676-9ab010cb2616 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:41.512824 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a9f7c-8ea9-48c4-b676-9ab010cb2616 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:41.517787 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f578689-1549-487d-bebd-26c13205e927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:41.518390 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f578689-1549-487d-bebd-26c13205e927 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:41.518671 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f578689-1549-487d-bebd-26c13205e927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:41.518949 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f578689-1549-487d-bebd-26c13205e927 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:41.519635 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f578689-1549-487d-bebd-26c13205e927 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:41.521559 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f578689-1549-487d-bebd-26c13205e927 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:41.536280 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 797/1582] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:51:41 2022] POST /identity/v3/users => generated 286 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:51:41.536280 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5bf5f-acda-40dd-8f55-d047f6f50901 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:41.546307 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5bf5f-acda-40dd-8f55-d047f6f50901 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:41.551575 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a5655-8728-4e88-aeb0-e2ebf1a4ac82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:41.552163 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a5655-8728-4e88-aeb0-e2ebf1a4ac82 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:41.552451 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a5655-8728-4e88-aeb0-e2ebf1a4ac82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:41.552736 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a5655-8728-4e88-aeb0-e2ebf1a4ac82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:41.553415 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a5655-8728-4e88-aeb0-e2ebf1a4ac82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:41.555012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a5655-8728-4e88-aeb0-e2ebf1a4ac82 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:41.565314 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 798/1583] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:51:41.566055 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a5655-8728-4e88-aeb0-e2ebf1a4ac82 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:41.575255 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a5655-8728-4e88-aeb0-e2ebf1a4ac82 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:41.579874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0acc8-3a70-4b53-9acc-8c5956ac8b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:41.580450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0acc8-3a70-4b53-9acc-8c5956ac8b06 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:41.580736 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0acc8-3a70-4b53-9acc-8c5956ac8b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:41.581014 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0acc8-3a70-4b53-9acc-8c5956ac8b06 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:41.581651 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0acc8-3a70-4b53-9acc-8c5956ac8b06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:41.583545 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0acc8-3a70-4b53-9acc-8c5956ac8b06 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:41.643362 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 786/1584] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:51:41 2022] POST /identity/v3/users => generated 287 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:51:41.643911 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f578689-1549-487d-bebd-26c13205e927 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:41.653205 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f578689-1549-487d-bebd-26c13205e927 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:41.658241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a147659-d855-407f-8a91-d055d1d52c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:41.658825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a147659-d855-407f-8a91-d055d1d52c46 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:41.659110 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a147659-d855-407f-8a91-d055d1d52c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:41.659381 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a147659-d855-407f-8a91-d055d1d52c46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:41.660096 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a147659-d855-407f-8a91-d055d1d52c46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:41.661683 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a147659-d855-407f-8a91-d055d1d52c46 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:41.672408 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 787/1585] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:51:41.673809 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a147659-d855-407f-8a91-d055d1d52c46 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:41.681384 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 799/1586] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:51:41 2022] POST /identity/v3/users => generated 287 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:51:41.688383 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0acc8-3a70-4b53-9acc-8c5956ac8b06 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:41.688515 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a147659-d855-407f-8a91-d055d1d52c46 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:41.690632 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9dcfe6-64d4-485e-b4f8-de3b7c91df32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:41.691246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dcfe6-64d4-485e-b4f8-de3b7c91df32 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:41.691564 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dcfe6-64d4-485e-b4f8-de3b7c91df32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:41.691841 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dcfe6-64d4-485e-b4f8-de3b7c91df32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:41.692040 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0acc8-3a70-4b53-9acc-8c5956ac8b06 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:41.692530 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9dcfe6-64d4-485e-b4f8-de3b7c91df32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:41.694154 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9dcfe6-64d4-485e-b4f8-de3b7c91df32 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:41.696763 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516cbac-a07a-46dc-b3a4-33f06d0a0475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:41.697360 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516cbac-a07a-46dc-b3a4-33f06d0a0475 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:41.697645 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516cbac-a07a-46dc-b3a4-33f06d0a0475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:41.697922 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516cbac-a07a-46dc-b3a4-33f06d0a0475 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:41.698611 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9516cbac-a07a-46dc-b3a4-33f06d0a0475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:41.700236 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9516cbac-a07a-46dc-b3a4-33f06d0a0475 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:41.707542 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 788/1587] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:51:41.707542 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9dcfe6-64d4-485e-b4f8-de3b7c91df32 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:41.707542 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2c9dcfe6-64d4-485e-b4f8-de3b7c91df32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:41.710511 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 800/1588] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:51:41.711148 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516cbac-a07a-46dc-b3a4-33f06d0a0475 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:41.721172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516cbac-a07a-46dc-b3a4-33f06d0a0475 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:41.725895 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0505942-cb7c-4693-a679-44fb6dca1d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:41.726497 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0505942-cb7c-4693-a679-44fb6dca1d8d admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:41.726804 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0505942-cb7c-4693-a679-44fb6dca1d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:41.727088 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0505942-cb7c-4693-a679-44fb6dca1d8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:41.727828 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0505942-cb7c-4693-a679-44fb6dca1d8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:41.729414 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0505942-cb7c-4693-a679-44fb6dca1d8d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:41.740359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 801/1589] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:51:41.741139 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0505942-cb7c-4693-a679-44fb6dca1d8d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:41.754589 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0505942-cb7c-4693-a679-44fb6dca1d8d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:41.759368 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fafc4f-b8b6-47c2-853c-22c2b1772fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:41.760052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fafc4f-b8b6-47c2-853c-22c2b1772fd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:41.760366 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fafc4f-b8b6-47c2-853c-22c2b1772fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:41.760642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fafc4f-b8b6-47c2-853c-22c2b1772fd9 admin admin] PATH_INFO: `/v3/projects/ca1bc66d9e5446758a759063544ae3e7/users/713d2a0f610d4296887b6ab9664ae5e6/roles/e033961a9363456a8510c34414dd839e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:41.769047 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9dcfe6-64d4-485e-b4f8-de3b7c91df32 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:41.772965 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451a1d5-b885-4aed-9da1-2179dd176983 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:41.773589 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451a1d5-b885-4aed-9da1-2179dd176983 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:41.773871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451a1d5-b885-4aed-9da1-2179dd176983 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:41.774150 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451a1d5-b885-4aed-9da1-2179dd176983 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:41.775771 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fafc4f-b8b6-47c2-853c-22c2b1772fd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca1bc66d9e5446758a759063544ae3e7, user_id=713d2a0f610d4296887b6ab9664ae5e6, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:41.778629 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fafc4f-b8b6-47c2-853c-22c2b1772fd9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:41.784192 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451a1d5-b885-4aed-9da1-2179dd176983 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:41.785683 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451a1d5-b885-4aed-9da1-2179dd176983 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:41.795358 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 789/1590] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:51:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:51:41.796168 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451a1d5-b885-4aed-9da1-2179dd176983 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:41.805214 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451a1d5-b885-4aed-9da1-2179dd176983 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:41.809821 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f2207-88d8-49ea-baed-b2856774be62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:41.810425 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f2207-88d8-49ea-baed-b2856774be62 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:41.810709 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f2207-88d8-49ea-baed-b2856774be62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:41.810988 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f2207-88d8-49ea-baed-b2856774be62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:41.811698 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02f2207-88d8-49ea-baed-b2856774be62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:41.813275 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02f2207-88d8-49ea-baed-b2856774be62 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:41.824000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 790/1591] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:51:41.824806 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f2207-88d8-49ea-baed-b2856774be62 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:41.834122 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f2207-88d8-49ea-baed-b2856774be62 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:41.838677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8df67-5cc2-43de-8de3-6971fe88040d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:41.839257 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8df67-5cc2-43de-8de3-6971fe88040d admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:41.839575 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8df67-5cc2-43de-8de3-6971fe88040d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:41.839853 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8df67-5cc2-43de-8de3-6971fe88040d admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:41.840522 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8df67-5cc2-43de-8de3-6971fe88040d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:41.842117 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8df67-5cc2-43de-8de3-6971fe88040d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:41.852160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 791/1592] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:51:41.852918 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8df67-5cc2-43de-8de3-6971fe88040d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:41.862614 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8df67-5cc2-43de-8de3-6971fe88040d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:41.867134 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c9e0f-adc2-4a05-8430-a6cc7da36240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:41.867730 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c9e0f-adc2-4a05-8430-a6cc7da36240 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:41.868268 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c9e0f-adc2-4a05-8430-a6cc7da36240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:41.868552 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c9e0f-adc2-4a05-8430-a6cc7da36240 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:41.869202 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c9e0f-adc2-4a05-8430-a6cc7da36240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:41.870724 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c9e0f-adc2-4a05-8430-a6cc7da36240 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:41.881003 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 792/1593] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:51:41.881766 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c9e0f-adc2-4a05-8430-a6cc7da36240 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:41.890798 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c9e0f-adc2-4a05-8430-a6cc7da36240 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:41.895914 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385dde4b-773e-4bb3-aa86-99e47cfebc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:41.896423 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385dde4b-773e-4bb3-aa86-99e47cfebc0f admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:41.896706 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385dde4b-773e-4bb3-aa86-99e47cfebc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:41.896980 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385dde4b-773e-4bb3-aa86-99e47cfebc0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:41.897638 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385dde4b-773e-4bb3-aa86-99e47cfebc0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:41.899160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385dde4b-773e-4bb3-aa86-99e47cfebc0f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:41.908753 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 793/1594] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:51:41.909533 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385dde4b-773e-4bb3-aa86-99e47cfebc0f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:41.918990 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385dde4b-773e-4bb3-aa86-99e47cfebc0f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:41.924192 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175b3fa-e884-4ef3-beea-f3e77bb2435a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:41.926305 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175b3fa-e884-4ef3-beea-f3e77bb2435a admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:41.926305 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175b3fa-e884-4ef3-beea-f3e77bb2435a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:41.926305 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175b3fa-e884-4ef3-beea-f3e77bb2435a admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:41.926305 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175b3fa-e884-4ef3-beea-f3e77bb2435a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:41.927532 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175b3fa-e884-4ef3-beea-f3e77bb2435a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:41.937156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 794/1595] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:51:41.938044 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175b3fa-e884-4ef3-beea-f3e77bb2435a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:41.947212 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175b3fa-e884-4ef3-beea-f3e77bb2435a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:41.952022 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446d4e1-405e-49b2-b401-05fe2876914b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:41.952627 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446d4e1-405e-49b2-b401-05fe2876914b admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:41.952941 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446d4e1-405e-49b2-b401-05fe2876914b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:41.953223 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446d4e1-405e-49b2-b401-05fe2876914b admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:41.953888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d446d4e1-405e-49b2-b401-05fe2876914b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:41.955980 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d446d4e1-405e-49b2-b401-05fe2876914b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:41.966249 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 795/1596] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:51:41.968444 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446d4e1-405e-49b2-b401-05fe2876914b admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:41.974277 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 802/1597] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:51:41 2022] PUT /identity/v3/projects/ca1bc66d9e5446758a759063544ae3e7/users/713d2a0f610d4296887b6ab9664ae5e6/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:51:41.974277 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fafc4f-b8b6-47c2-853c-22c2b1772fd9 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:41.978478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446d4e1-405e-49b2-b401-05fe2876914b admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:42.047889 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:51:42.056276 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fafc4f-b8b6-47c2-853c-22c2b1772fd9 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:42.061406 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b129d-5bc7-497f-9f0f-1559bdaa6a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:42.061577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4a975-e124-46aa-a049-e9c0865b763f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:42.063852 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4a975-e124-46aa-a049-e9c0865b763f admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:42.064012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b129d-5bc7-497f-9f0f-1559bdaa6a1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:42.064632 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4a975-e124-46aa-a049-e9c0865b763f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:42.064745 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b129d-5bc7-497f-9f0f-1559bdaa6a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:42.064845 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4a975-e124-46aa-a049-e9c0865b763f admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:42.064936 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b129d-5bc7-497f-9f0f-1559bdaa6a1e admin admin] PATH_INFO: `/v3/projects/4f3f79533bbf48018798e6e65050d69c/users/19c0408b4321495eabd4738bbb8b711b/roles/e033961a9363456a8510c34414dd839e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:42.065025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4a975-e124-46aa-a049-e9c0865b763f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:42.065645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4a975-e124-46aa-a049-e9c0865b763f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:42.079820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 796/1598] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:51:42.079820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4a975-e124-46aa-a049-e9c0865b763f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:42.087091 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5b129d-5bc7-497f-9f0f-1559bdaa6a1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f3f79533bbf48018798e6e65050d69c, user_id=19c0408b4321495eabd4738bbb8b711b, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:42.090631 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5b129d-5bc7-497f-9f0f-1559bdaa6a1e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:42.090806 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4a975-e124-46aa-a049-e9c0865b763f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:42.095587 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa05fd0-7158-46e0-986c-6df8c97654a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:42.096218 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa05fd0-7158-46e0-986c-6df8c97654a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:42.096502 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa05fd0-7158-46e0-986c-6df8c97654a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:42.096779 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa05fd0-7158-46e0-986c-6df8c97654a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:42.097536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa05fd0-7158-46e0-986c-6df8c97654a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:42.099369 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa05fd0-7158-46e0-986c-6df8c97654a9 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:42.110268 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 797/1599] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:51:42.111735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa05fd0-7158-46e0-986c-6df8c97654a9 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:42.122158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa05fd0-7158-46e0-986c-6df8c97654a9 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:42.127321 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd4a05-fbf1-4938-b031-1663d25e3151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:42.127954 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd4a05-fbf1-4938-b031-1663d25e3151 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:42.128267 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd4a05-fbf1-4938-b031-1663d25e3151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:42.128545 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd4a05-fbf1-4938-b031-1663d25e3151 admin admin] PATH_INFO: `/v3/projects/09d5ca5e2ba947e7acc7ceb298e2aa85/users/239fed388926425fa8987ad9fac79394/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:42.144382 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd4a05-fbf1-4938-b031-1663d25e3151 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09d5ca5e2ba947e7acc7ceb298e2aa85, user_id=239fed388926425fa8987ad9fac79394, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:42.147123 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd4a05-fbf1-4938-b031-1663d25e3151 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:42.202401 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 803/1600] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:51:41 2022] PUT /identity/v3/projects/4f3f79533bbf48018798e6e65050d69c/users/19c0408b4321495eabd4738bbb8b711b/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:51:42.203215 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b129d-5bc7-497f-9f0f-1559bdaa6a1e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:42.241466 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:51:42.269324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b129d-5bc7-497f-9f0f-1559bdaa6a1e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:42.273986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0d78a-29f0-495c-8a6e-ef746dd5a802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:42.274665 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0d78a-29f0-495c-8a6e-ef746dd5a802 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:42.274953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0d78a-29f0-495c-8a6e-ef746dd5a802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:42.275231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0d78a-29f0-495c-8a6e-ef746dd5a802 admin admin] PATH_INFO: `/v3/projects/7ccd4ff518dc4c37bfb7f343ccc97206/users/e0615d20057342cfbd16aa0ec94d1979/roles/e033961a9363456a8510c34414dd839e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:42.292839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0d78a-29f0-495c-8a6e-ef746dd5a802 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ccd4ff518dc4c37bfb7f343ccc97206, user_id=e0615d20057342cfbd16aa0ec94d1979, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:42.300973 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0d78a-29f0-495c-8a6e-ef746dd5a802 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:42.307694 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 798/1601] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:51:42 2022] PUT /identity/v3/projects/09d5ca5e2ba947e7acc7ceb298e2aa85/users/239fed388926425fa8987ad9fac79394/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:51:42.309237 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd4a05-fbf1-4938-b031-1663d25e3151 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:42.371572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd4a05-fbf1-4938-b031-1663d25e3151 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:42.376967 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3664ef-75e4-4a8f-b1bb-0ab8dc9816a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:42.377662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3664ef-75e4-4a8f-b1bb-0ab8dc9816a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:42.377946 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3664ef-75e4-4a8f-b1bb-0ab8dc9816a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:42.378224 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3664ef-75e4-4a8f-b1bb-0ab8dc9816a1 admin admin] PATH_INFO: `/v3/projects/95beb7e102724f2d9172f68a61a65c7e/users/299aa937a2004ac181685cbc03ab8146/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:42.386920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 804/1602] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:51:42 2022] PUT /identity/v3/projects/7ccd4ff518dc4c37bfb7f343ccc97206/users/e0615d20057342cfbd16aa0ec94d1979/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:51:42.388310 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be0cfd-891d-4264-97e4-5a0f106e9aab None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:42.388933 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be0cfd-891d-4264-97e4-5a0f106e9aab None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:42.388933 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be0cfd-891d-4264-97e4-5a0f106e9aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:42.395733 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3664ef-75e4-4a8f-b1bb-0ab8dc9816a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95beb7e102724f2d9172f68a61a65c7e, user_id=299aa937a2004ac181685cbc03ab8146, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:42.398588 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3664ef-75e4-4a8f-b1bb-0ab8dc9816a1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:42.408315 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f3be0cfd-891d-4264-97e4-5a0f106e9aab None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:42.409545 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f3be0cfd-891d-4264-97e4-5a0f106e9aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:42.472855 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 799/1603] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:51:42 2022] PUT /identity/v3/projects/95beb7e102724f2d9172f68a61a65c7e/users/299aa937a2004ac181685cbc03ab8146/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:51:42.473995 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3664ef-75e4-4a8f-b1bb-0ab8dc9816a1 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:42.482499 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 805/1604] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:42.485115 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be0cfd-891d-4264-97e4-5a0f106e9aab None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:42.497854 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be0cfd-891d-4264-97e4-5a0f106e9aab None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:42.569132 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f29a7e-9cea-4788-8bc4-b1307405338c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:42.569810 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f29a7e-9cea-4788-8bc4-b1307405338c admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:42.570094 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f29a7e-9cea-4788-8bc4-b1307405338c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:42.576293 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f29a7e-9cea-4788-8bc4-b1307405338c admin admin] PATH_INFO: `/v3/projects/6a4d0de164894881bcec30ff989a283b/users/904ae63d554f4435ad5f7702bea2d3ba/roles/e033961a9363456a8510c34414dd839e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:42.579244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3664ef-75e4-4a8f-b1bb-0ab8dc9816a1 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:42.584613 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019f2ef-ac0b-4976-b488-7213a9bc7f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:42.585286 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019f2ef-ac0b-4976-b488-7213a9bc7f4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:42.585641 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019f2ef-ac0b-4976-b488-7213a9bc7f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:42.586178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019f2ef-ac0b-4976-b488-7213a9bc7f4b admin admin] PATH_INFO: `/v3/projects/2f0022e5d6334556bca1b43bce6f6ecf/users/266b64f12ac44348aaa2742a16dd4b1e/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:42.586699 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f29a7e-9cea-4788-8bc4-b1307405338c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a4d0de164894881bcec30ff989a283b, user_id=904ae63d554f4435ad5f7702bea2d3ba, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:42.589593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f29a7e-9cea-4788-8bc4-b1307405338c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:42.605454 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019f2ef-ac0b-4976-b488-7213a9bc7f4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f0022e5d6334556bca1b43bce6f6ecf, user_id=266b64f12ac44348aaa2742a16dd4b1e, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:42.608543 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019f2ef-ac0b-4976-b488-7213a9bc7f4b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:42.731482 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 806/1605] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:51:42 2022] PUT /identity/v3/projects/6a4d0de164894881bcec30ff989a283b/users/904ae63d554f4435ad5f7702bea2d3ba/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:51:42.731748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 800/1606] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:51:42 2022] PUT /identity/v3/projects/2f0022e5d6334556bca1b43bce6f6ecf/users/266b64f12ac44348aaa2742a16dd4b1e/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:51:42.731867 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4af87-e481-4963-ac11-2a3cc3cb4d2f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:42.731960 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad80af6-c5f5-449f-9f42-616f612c95c3 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:42.731960 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad80af6-c5f5-449f-9f42-616f612c95c3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:42.732111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4af87-e481-4963-ac11-2a3cc3cb4d2f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:42.732199 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad80af6-c5f5-449f-9f42-616f612c95c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:42.732301 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4af87-e481-4963-ac11-2a3cc3cb4d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:42.755758 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-67d4af87-e481-4963-ac11-2a3cc3cb4d2f None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:42.757116 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-67d4af87-e481-4963-ac11-2a3cc3cb4d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:42.771324 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-bad80af6-c5f5-449f-9f42-616f612c95c3 None None] MFA Rules not processed for user `239fed388926425fa8987ad9fac79394`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:42.772575 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-bad80af6-c5f5-449f-9f42-616f612c95c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:42.783342 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:51:42.796360 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:51:42.828002 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 807/1607] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:42.828002 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13356fa-deb8-413e-9010-9171db2817ef None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:42.828002 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13356fa-deb8-413e-9010-9171db2817ef None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:42.828002 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13356fa-deb8-413e-9010-9171db2817ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:42.851927 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d13356fa-deb8-413e-9010-9171db2817ef None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:42.853232 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d13356fa-deb8-413e-9010-9171db2817ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:42.857185 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 801/1608] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:42.858683 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911aaa8-e148-4555-bf0e-713d6ce099a9 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:42.858978 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911aaa8-e148-4555-bf0e-713d6ce099a9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:42.859255 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911aaa8-e148-4555-bf0e-713d6ce099a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:42.864648 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 808/1609] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:42.867696 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13356fa-deb8-413e-9010-9171db2817ef None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:42.896293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b911aaa8-e148-4555-bf0e-713d6ce099a9 None None] MFA Rules not processed for user `299aa937a2004ac181685cbc03ab8146`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:42.897798 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b911aaa8-e148-4555-bf0e-713d6ce099a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:42.939526 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13356fa-deb8-413e-9010-9171db2817ef None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:42.945908 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a6ea0-ffba-4e9e-8f83-d56317930981 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:42.945908 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a6ea0-ffba-4e9e-8f83-d56317930981 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:42.945908 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a6ea0-ffba-4e9e-8f83-d56317930981 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:42.946501 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a6ea0-ffba-4e9e-8f83-d56317930981 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:42.957612 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a6ea0-ffba-4e9e-8f83-d56317930981 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:42.959530 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a6ea0-ffba-4e9e-8f83-d56317930981 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:42.969008 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 809/1610] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:51:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:51:42.970416 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7d9b4-e284-4d0c-b75f-5758762e523e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:42.970710 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7d9b4-e284-4d0c-b75f-5758762e523e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:42.970986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7d9b4-e284-4d0c-b75f-5758762e523e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:42.973429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 802/1611] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:42.975145 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e529346-ba2f-4677-8854-9e0c3e941345 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:42.975499 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e529346-ba2f-4677-8854-9e0c3e941345 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:42.975782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e529346-ba2f-4677-8854-9e0c3e941345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:43.005798 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-07b7d9b4-e284-4d0c-b75f-5758762e523e None None] MFA Rules not processed for user `266b64f12ac44348aaa2742a16dd4b1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:43.007445 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-07b7d9b4-e284-4d0c-b75f-5758762e523e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:43.012522 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-7e529346-ba2f-4677-8854-9e0c3e941345 None None] MFA Rules not processed for user `904ae63d554f4435ad5f7702bea2d3ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:43.013655 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7e529346-ba2f-4677-8854-9e0c3e941345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:43.091849 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 810/1612] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:43.091849 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b7d9b4-e284-4d0c-b75f-5758762e523e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:43.097859 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b7d9b4-e284-4d0c-b75f-5758762e523e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:43.102912 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c3a19-5db9-4160-9498-269a2c652d6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:43.103811 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c3a19-5db9-4160-9498-269a2c652d6d service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:43.104838 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c3a19-5db9-4160-9498-269a2c652d6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:43.105224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c3a19-5db9-4160-9498-269a2c652d6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:43.119108 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c3a19-5db9-4160-9498-269a2c652d6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:43.121722 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c3a19-5db9-4160-9498-269a2c652d6d service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:43.128196 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 803/1613] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:43.128196 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e529346-ba2f-4677-8854-9e0c3e941345 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:43.134223 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 811/1614] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:51:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:51:43.135105 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c3a19-5db9-4160-9498-269a2c652d6d service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:43.145378 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e529346-ba2f-4677-8854-9e0c3e941345 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:43.145606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c3a19-5db9-4160-9498-269a2c652d6d service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:43.149804 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963cf2ec-6a6d-4845-8145-85a86838f44f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:43.150446 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a94291-7212-4831-81e0-ef26715fc183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:43.150584 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963cf2ec-6a6d-4845-8145-85a86838f44f service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:43.150975 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963cf2ec-6a6d-4845-8145-85a86838f44f service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:43.151290 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a94291-7212-4831-81e0-ef26715fc183 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:43.151393 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963cf2ec-6a6d-4845-8145-85a86838f44f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:43.151728 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a94291-7212-4831-81e0-ef26715fc183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:43.152072 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a94291-7212-4831-81e0-ef26715fc183 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:43.152956 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a94291-7212-4831-81e0-ef26715fc183 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:43.154945 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a94291-7212-4831-81e0-ef26715fc183 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:43.164064 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963cf2ec-6a6d-4845-8145-85a86838f44f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:43.165543 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963cf2ec-6a6d-4845-8145-85a86838f44f service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:43.179654 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 812/1615] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:51:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:51:43.180555 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963cf2ec-6a6d-4845-8145-85a86838f44f service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:43.190824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963cf2ec-6a6d-4845-8145-85a86838f44f service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:43.195372 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb2198c-c020-40f0-ba31-cf169903a764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:43.196093 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2198c-c020-40f0-ba31-cf169903a764 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:43.196381 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2198c-c020-40f0-ba31-cf169903a764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:43.196663 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2198c-c020-40f0-ba31-cf169903a764 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:43.197328 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb2198c-c020-40f0-ba31-cf169903a764 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:43.199248 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb2198c-c020-40f0-ba31-cf169903a764 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:43.256622 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 804/1616] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:51:43 2022] POST /identity/v3/projects => generated 379 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:51:43.257516 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a94291-7212-4831-81e0-ef26715fc183 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:43.266757 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:51:43.317258 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a94291-7212-4831-81e0-ef26715fc183 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:43.320040 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 813/1617] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:51:43 2022] POST /identity/v3/projects => generated 371 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:51:43.321386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be325d-4f57-422e-9482-9b0d71d396b2 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:43.321678 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be325d-4f57-422e-9482-9b0d71d396b2 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:43.321957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be325d-4f57-422e-9482-9b0d71d396b2 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:43.323189 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 814/1618] 10.222.0.97 () {54 vars in 962 bytes} [Tue Jul 26 14:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:51:43.325262 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd7b54-9836-49eb-bbda-d61447ed11e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:43.325960 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd7b54-9836-49eb-bbda-d61447ed11e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:43.326248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd7b54-9836-49eb-bbda-d61447ed11e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:43.326523 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd7b54-9836-49eb-bbda-d61447ed11e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:43.327246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd7b54-9836-49eb-bbda-d61447ed11e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:43.328120 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be325d-4f57-422e-9482-9b0d71d396b2 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:43.329365 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd7b54-9836-49eb-bbda-d61447ed11e0 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:43.367258 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:51:43.427871 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be325d-4f57-422e-9482-9b0d71d396b2 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:43.433870 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f6edb7-549b-4dca-9490-5935491a645e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:43.434661 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6edb7-549b-4dca-9490-5935491a645e admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:43.435077 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6edb7-549b-4dca-9490-5935491a645e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:43.435367 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6edb7-549b-4dca-9490-5935491a645e admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:43.436173 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f6edb7-549b-4dca-9490-5935491a645e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:43.438194 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f6edb7-549b-4dca-9490-5935491a645e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:43.494539 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 805/1619] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:51:43 2022] POST /identity/v3/projects => generated 373 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:51:43.495363 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd7b54-9836-49eb-bbda-d61447ed11e0 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:43.581226 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 815/1620] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:51:43 2022] POST /identity/v3/users => generated 296 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:51:43.584204 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde41a7-25a8-4a28-b5f3-dbd67fe5a81a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:43.584692 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde41a7-25a8-4a28-b5f3-dbd67fe5a81a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:43.585128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde41a7-25a8-4a28-b5f3-dbd67fe5a81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:43.594173 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd7b54-9836-49eb-bbda-d61447ed11e0 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:43.606698 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be9b65f-0a13-4d2e-9dad-76ca3e04568c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:43.607938 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9b65f-0a13-4d2e-9dad-76ca3e04568c admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:43.608378 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9b65f-0a13-4d2e-9dad-76ca3e04568c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:43.609108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9b65f-0a13-4d2e-9dad-76ca3e04568c admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:43.610682 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be9b65f-0a13-4d2e-9dad-76ca3e04568c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:43.616329 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be9b65f-0a13-4d2e-9dad-76ca3e04568c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:43.644455 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-9cde41a7-25a8-4a28-b5f3-dbd67fe5a81a None None] MFA Rules not processed for user `266abef6cda5495b897170f92043aa03`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:43.645817 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9cde41a7-25a8-4a28-b5f3-dbd67fe5a81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:43.722694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 816/1621] 10.222.0.97 () {58 vars in 1059 bytes} [Tue Jul 26 14:51:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:43.728432 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde41a7-25a8-4a28-b5f3-dbd67fe5a81a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:43.740763 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde41a7-25a8-4a28-b5f3-dbd67fe5a81a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:43.746284 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a3116e-1451-4570-9380-4c7e1a75cc5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:43.747963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3116e-1451-4570-9380-4c7e1a75cc5d admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:43.748643 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3116e-1451-4570-9380-4c7e1a75cc5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:43.750108 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3116e-1451-4570-9380-4c7e1a75cc5d admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:43.750915 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a3116e-1451-4570-9380-4c7e1a75cc5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:43.753225 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a3116e-1451-4570-9380-4c7e1a75cc5d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:43.754347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 806/1622] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:51:43 2022] POST /identity/v3/users => generated 292 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:51:43.755423 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be9b65f-0a13-4d2e-9dad-76ca3e04568c admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:43.770228 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be9b65f-0a13-4d2e-9dad-76ca3e04568c admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:43.776432 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7efc1-c63d-4adb-8150-762db915ef19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:43.776742 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7efc1-c63d-4adb-8150-762db915ef19 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:43.777004 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7efc1-c63d-4adb-8150-762db915ef19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:43.777283 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7efc1-c63d-4adb-8150-762db915ef19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:43.778741 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c7efc1-c63d-4adb-8150-762db915ef19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:43.780378 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c7efc1-c63d-4adb-8150-762db915ef19 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:43.801674 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 807/1623] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:51:43.802681 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7efc1-c63d-4adb-8150-762db915ef19 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:43.815273 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7efc1-c63d-4adb-8150-762db915ef19 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:43.821481 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c638b-d76c-49f7-82e8-0733b6b1beef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:43.822118 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c638b-d76c-49f7-82e8-0733b6b1beef service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:43.822403 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c638b-d76c-49f7-82e8-0733b6b1beef service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:43.823712 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c638b-d76c-49f7-82e8-0733b6b1beef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:43.907620 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 817/1624] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:51:43 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:51:43.911171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a3116e-1451-4570-9380-4c7e1a75cc5d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:43.931715 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a3116e-1451-4570-9380-4c7e1a75cc5d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:43.944553 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483b5ad-afff-4def-a1a0-9595657a1a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:43.948151 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483b5ad-afff-4def-a1a0-9595657a1a5a admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:43.948151 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483b5ad-afff-4def-a1a0-9595657a1a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:43.948151 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483b5ad-afff-4def-a1a0-9595657a1a5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:43.948151 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2483b5ad-afff-4def-a1a0-9595657a1a5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:43.950901 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2483b5ad-afff-4def-a1a0-9595657a1a5a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:43.954825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c638b-d76c-49f7-82e8-0733b6b1beef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:43.956254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c638b-d76c-49f7-82e8-0733b6b1beef service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:43.991342 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 808/1625] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:51:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:51:43.996427 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 818/1626] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:51:43.996427 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483b5ad-afff-4def-a1a0-9595657a1a5a admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:43.996654 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c638b-d76c-49f7-82e8-0733b6b1beef service nova] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:44.003723 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483b5ad-afff-4def-a1a0-9595657a1a5a admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:44.011963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f1587-af70-47e0-bbab-f04462c34b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:44.015563 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f1587-af70-47e0-bbab-f04462c34b6f admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:44.015873 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f1587-af70-47e0-bbab-f04462c34b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:44.015984 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c638b-d76c-49f7-82e8-0733b6b1beef service nova] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:44.016132 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f1587-af70-47e0-bbab-f04462c34b6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:44.016851 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f1587-af70-47e0-bbab-f04462c34b6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:44.018813 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f1587-af70-47e0-bbab-f04462c34b6f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:44.048132 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1659d4-285c-4c9c-bbf5-613d12387a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:44.048132 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1659d4-285c-4c9c-bbf5-613d12387a89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:44.048132 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1659d4-285c-4c9c-bbf5-613d12387a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:44.048132 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1659d4-285c-4c9c-bbf5-613d12387a89 admin admin] PATH_INFO: `/v3/projects/005a74bb12454f9b99ac6074ed3f18a9/users/b387682a85884879a8cd21b3c7e70cf8/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:44.052720 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:51:44.053665 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 819/1627] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:51:44.056354 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f1587-af70-47e0-bbab-f04462c34b6f admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:44.069848 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f1587-af70-47e0-bbab-f04462c34b6f admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:44.077857 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1659d4-285c-4c9c-bbf5-613d12387a89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=005a74bb12454f9b99ac6074ed3f18a9, user_id=b387682a85884879a8cd21b3c7e70cf8, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:44.080325 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e74f10-e8c3-4b2b-8901-36ad90b29ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:44.080669 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1659d4-285c-4c9c-bbf5-613d12387a89 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:44.082130 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e74f10-e8c3-4b2b-8901-36ad90b29ddf admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:44.082671 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e74f10-e8c3-4b2b-8901-36ad90b29ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:44.083389 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e74f10-e8c3-4b2b-8901-36ad90b29ddf admin admin] PATH_INFO: `/v3/projects/7b6a8cb3b4174123963275fa10ad0565/users/741730a5c7934c7f9327831a7015c2c7/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:44.127994 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e74f10-e8c3-4b2b-8901-36ad90b29ddf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b6a8cb3b4174123963275fa10ad0565, user_id=741730a5c7934c7f9327831a7015c2c7, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:44.131428 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e74f10-e8c3-4b2b-8901-36ad90b29ddf admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:44.303613 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 820/1628] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:51:44 2022] PUT /identity/v3/projects/7b6a8cb3b4174123963275fa10ad0565/users/741730a5c7934c7f9327831a7015c2c7/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:51:44.304240 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e74f10-e8c3-4b2b-8901-36ad90b29ddf admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:44.313687 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 809/1629] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:51:43 2022] PUT /identity/v3/projects/005a74bb12454f9b99ac6074ed3f18a9/users/b387682a85884879a8cd21b3c7e70cf8/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:51:44.315507 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874ea09-b67e-448d-b370-3eae367a7564 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:44.316091 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874ea09-b67e-448d-b370-3eae367a7564 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:44.316624 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874ea09-b67e-448d-b370-3eae367a7564 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:44.318500 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 810/1630] 10.222.0.97 () {54 vars in 912 bytes} [Tue Jul 26 14:51:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:51:44.319957 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874ea09-b67e-448d-b370-3eae367a7564 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:44.339202 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:51:44.383091 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e74f10-e8c3-4b2b-8901-36ad90b29ddf admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:44.395997 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874ea09-b67e-448d-b370-3eae367a7564 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:44.401159 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0734d4-1b1c-4a66-b4ae-b9205ac79dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:44.402254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0734d4-1b1c-4a66-b4ae-b9205ac79dd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:44.402254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0734d4-1b1c-4a66-b4ae-b9205ac79dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:44.402450 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0734d4-1b1c-4a66-b4ae-b9205ac79dd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:44.403035 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0734d4-1b1c-4a66-b4ae-b9205ac79dd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:44.405279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0734d4-1b1c-4a66-b4ae-b9205ac79dd8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:44.419937 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 811/1631] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:51:44.419937 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0734d4-1b1c-4a66-b4ae-b9205ac79dd8 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:44.430138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0734d4-1b1c-4a66-b4ae-b9205ac79dd8 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:44.435147 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a180224-76fc-4cc4-adbd-f0fbe8a59a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:44.435816 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a180224-76fc-4cc4-adbd-f0fbe8a59a91 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:44.436105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a180224-76fc-4cc4-adbd-f0fbe8a59a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:44.436385 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a180224-76fc-4cc4-adbd-f0fbe8a59a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:44.437083 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a180224-76fc-4cc4-adbd-f0fbe8a59a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:44.438704 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a180224-76fc-4cc4-adbd-f0fbe8a59a91 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:44.449415 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df648f1d-b5d0-4061-90b4-0d23c8ad1ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:44.450221 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df648f1d-b5d0-4061-90b4-0d23c8ad1ca6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:44.450511 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df648f1d-b5d0-4061-90b4-0d23c8ad1ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:44.450848 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df648f1d-b5d0-4061-90b4-0d23c8ad1ca6 admin admin] PATH_INFO: `/v3/projects/d6e4536022c94225b14eba5d808bc436/users/73e4210496a14e1ba33768dec4a128eb/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:44.451143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 812/1632] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:51:44.456692 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47949936-5666-4349-9c31-c13f5030b836 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:44.456692 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47949936-5666-4349-9c31-c13f5030b836 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:44.456952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47949936-5666-4349-9c31-c13f5030b836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:44.470250 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df648f1d-b5d0-4061-90b4-0d23c8ad1ca6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6e4536022c94225b14eba5d808bc436, user_id=73e4210496a14e1ba33768dec4a128eb, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:44.473586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df648f1d-b5d0-4061-90b4-0d23c8ad1ca6 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:44.478212 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-47949936-5666-4349-9c31-c13f5030b836 None None] MFA Rules not processed for user `266abef6cda5495b897170f92043aa03`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:44.479482 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-47949936-5666-4349-9c31-c13f5030b836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:44.603528 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 813/1633] 10.222.0.97 () {58 vars in 1009 bytes} [Tue Jul 26 14:51:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:44.604623 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47949936-5666-4349-9c31-c13f5030b836 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:44.621488 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 821/1634] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:51:44 2022] PUT /identity/v3/projects/d6e4536022c94225b14eba5d808bc436/users/73e4210496a14e1ba33768dec4a128eb/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:51:44.626463 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df648f1d-b5d0-4061-90b4-0d23c8ad1ca6 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:44.626737 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47949936-5666-4349-9c31-c13f5030b836 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:44.721155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068e1ab-ff6c-46bd-84f6-dc9bd9b35ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:44.722307 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068e1ab-ff6c-46bd-84f6-dc9bd9b35ff3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:44.722877 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068e1ab-ff6c-46bd-84f6-dc9bd9b35ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:44.723386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068e1ab-ff6c-46bd-84f6-dc9bd9b35ff3 admin admin] PATH_INFO: `/v3/projects/7b6a8cb3b4174123963275fa10ad0565/users/741730a5c7934c7f9327831a7015c2c7/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:44.725822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068e1ab-ff6c-46bd-84f6-dc9bd9b35ff3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b6a8cb3b4174123963275fa10ad0565, user_id=741730a5c7934c7f9327831a7015c2c7, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:44.734645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068e1ab-ff6c-46bd-84f6-dc9bd9b35ff3 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:45.199120 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df648f1d-b5d0-4061-90b4-0d23c8ad1ca6 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:45.206145 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 814/1635] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:51:44 2022] PUT /identity/v3/projects/7b6a8cb3b4174123963275fa10ad0565/users/741730a5c7934c7f9327831a7015c2c7/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:51:45.206145 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068e1ab-ff6c-46bd-84f6-dc9bd9b35ff3 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:45.208158 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500a6cd-3120-4ea2-9244-7647b2731177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:45.208158 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500a6cd-3120-4ea2-9244-7647b2731177 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:45.208158 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500a6cd-3120-4ea2-9244-7647b2731177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:45.208158 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500a6cd-3120-4ea2-9244-7647b2731177 admin admin] PATH_INFO: `/v3/projects/005a74bb12454f9b99ac6074ed3f18a9/users/b387682a85884879a8cd21b3c7e70cf8/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:45.209671 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500a6cd-3120-4ea2-9244-7647b2731177 admin admin] RBAC: Authorizing `identity:create_grant(project_id=005a74bb12454f9b99ac6074ed3f18a9, user_id=b387682a85884879a8cd21b3c7e70cf8, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:45.209671 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500a6cd-3120-4ea2-9244-7647b2731177 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:45.209671 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 822/1636] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:51:44 2022] PUT /identity/v3/projects/005a74bb12454f9b99ac6074ed3f18a9/users/b387682a85884879a8cd21b3c7e70cf8/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:51:45.209671 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500a6cd-3120-4ea2-9244-7647b2731177 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:45.211605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068e1ab-ff6c-46bd-84f6-dc9bd9b35ff3 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:45.213346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500a6cd-3120-4ea2-9244-7647b2731177 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:45.213346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e530ab0b-1849-4584-9680-22f7b71b7152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:45.213346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530ab0b-1849-4584-9680-22f7b71b7152 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:45.213346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530ab0b-1849-4584-9680-22f7b71b7152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:45.215021 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530ab0b-1849-4584-9680-22f7b71b7152 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:45.215021 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e530ab0b-1849-4584-9680-22f7b71b7152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:45.215021 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e530ab0b-1849-4584-9680-22f7b71b7152 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:45.215021 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 823/1637] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:51:45.215021 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9375d-f6fc-42e9-83cf-57428aec7ff0 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:45.215021 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9375d-f6fc-42e9-83cf-57428aec7ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:45.219240 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9375d-f6fc-42e9-83cf-57428aec7ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:45.241882 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9651eb-67bc-4b88-9a75-3bd740e45aed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:45.241882 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9651eb-67bc-4b88-9a75-3bd740e45aed service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:45.241882 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9651eb-67bc-4b88-9a75-3bd740e45aed service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:45.241882 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9651eb-67bc-4b88-9a75-3bd740e45aed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:45.242561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-86a9375d-f6fc-42e9-83cf-57428aec7ff0 None None] MFA Rules not processed for user `741730a5c7934c7f9327831a7015c2c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:45.242561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-86a9375d-f6fc-42e9-83cf-57428aec7ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:45.242561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 824/1638] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:44 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:45.242561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c495ce9-0f60-44e0-936b-a4c3ce23d17a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:45.242561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c495ce9-0f60-44e0-936b-a4c3ce23d17a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:45.242561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c495ce9-0f60-44e0-936b-a4c3ce23d17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:45.243026 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9651eb-67bc-4b88-9a75-3bd740e45aed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:45.243026 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9651eb-67bc-4b88-9a75-3bd740e45aed service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:45.243231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3c495ce9-0f60-44e0-936b-a4c3ce23d17a None None] MFA Rules not processed for user `b387682a85884879a8cd21b3c7e70cf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:45.243231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3c495ce9-0f60-44e0-936b-a4c3ce23d17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:45.243536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 815/1639] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:51:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:51:45.243536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9651eb-67bc-4b88-9a75-3bd740e45aed service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:45.243536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9651eb-67bc-4b88-9a75-3bd740e45aed service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:45.243536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c60ed-d106-49a4-9b6c-517c0160ae69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:45.244076 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c60ed-d106-49a4-9b6c-517c0160ae69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:45.244076 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c60ed-d106-49a4-9b6c-517c0160ae69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:45.244076 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c60ed-d106-49a4-9b6c-517c0160ae69 admin admin] PATH_INFO: `/v3/projects/d6e4536022c94225b14eba5d808bc436/users/73e4210496a14e1ba33768dec4a128eb/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:45.244076 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99c60ed-d106-49a4-9b6c-517c0160ae69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6e4536022c94225b14eba5d808bc436, user_id=73e4210496a14e1ba33768dec4a128eb, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:45.244076 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99c60ed-d106-49a4-9b6c-517c0160ae69 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:45.245161 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 816/1640] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:51:45 2022] PUT /identity/v3/projects/d6e4536022c94225b14eba5d808bc436/users/73e4210496a14e1ba33768dec4a128eb/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:51:45.245161 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c60ed-d106-49a4-9b6c-517c0160ae69 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:45.319249 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 825/1641] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:45 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:45.326381 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675880c-1c52-4f3f-add8-d1c9b13eb11d None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:45.326680 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675880c-1c52-4f3f-add8-d1c9b13eb11d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:45.326967 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675880c-1c52-4f3f-add8-d1c9b13eb11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:45.350501 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c60ed-d106-49a4-9b6c-517c0160ae69 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:45.361789 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2ce77-6af7-4b3a-9f9f-64b32438b560 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:45.362438 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2ce77-6af7-4b3a-9f9f-64b32438b560 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:45.362723 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2ce77-6af7-4b3a-9f9f-64b32438b560 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:45.363003 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2ce77-6af7-4b3a-9f9f-64b32438b560 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:45.392259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-8675880c-1c52-4f3f-add8-d1c9b13eb11d None None] MFA Rules not processed for user `73e4210496a14e1ba33768dec4a128eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:45.397102 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8675880c-1c52-4f3f-add8-d1c9b13eb11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:45.497320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c2ce77-6af7-4b3a-9f9f-64b32438b560 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:45.516988 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c2ce77-6af7-4b3a-9f9f-64b32438b560 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:45.517121 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 826/1642] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:45 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:45.517208 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 817/1643] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:51:45 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:51:45.730532 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675880c-1c52-4f3f-add8-d1c9b13eb11d None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:45.741954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675880c-1c52-4f3f-add8-d1c9b13eb11d None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:45.746817 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c2e25-b9c8-43db-8a21-9e75d4304618 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:45.747277 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c2e25-b9c8-43db-8a21-9e75d4304618 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:45.747603 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c2e25-b9c8-43db-8a21-9e75d4304618 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:45.747933 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c2e25-b9c8-43db-8a21-9e75d4304618 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:45.844680 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c2e25-b9c8-43db-8a21-9e75d4304618 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:45.846416 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c2e25-b9c8-43db-8a21-9e75d4304618 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:45.862829 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 827/1644] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:51:45 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:51:46.005313 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2ce77-6af7-4b3a-9f9f-64b32438b560 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:46.019984 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2ce77-6af7-4b3a-9f9f-64b32438b560 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:46.029047 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd32ae2-c57b-4b27-b7cc-7c106c38716a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:46.030137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd32ae2-c57b-4b27-b7cc-7c106c38716a service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:46.030673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd32ae2-c57b-4b27-b7cc-7c106c38716a service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:46.031192 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd32ae2-c57b-4b27-b7cc-7c106c38716a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:46.047643 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd32ae2-c57b-4b27-b7cc-7c106c38716a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:46.050961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd32ae2-c57b-4b27-b7cc-7c106c38716a service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:46.079595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 818/1645] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:51:46 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:51:46.188482 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:51:46.432894 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:51:46.696965 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c2e25-b9c8-43db-8a21-9e75d4304618 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:46.723325 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c2e25-b9c8-43db-8a21-9e75d4304618 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:46.739363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2627974d-2982-42e5-a8e2-81d3d3d335cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:46.739896 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627974d-2982-42e5-a8e2-81d3d3d335cb service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:46.741188 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627974d-2982-42e5-a8e2-81d3d3d335cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:46.741576 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627974d-2982-42e5-a8e2-81d3d3d335cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:46.868103 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd32ae2-c57b-4b27-b7cc-7c106c38716a service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:46.880043 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd32ae2-c57b-4b27-b7cc-7c106c38716a service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:46.897734 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1312a4c-2266-4098-b71d-3aa7b9929747 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:46.898607 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1312a4c-2266-4098-b71d-3aa7b9929747 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:46.899025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1312a4c-2266-4098-b71d-3aa7b9929747 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:46.899689 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1312a4c-2266-4098-b71d-3aa7b9929747 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:46.949492 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2627974d-2982-42e5-a8e2-81d3d3d335cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:46.952121 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2627974d-2982-42e5-a8e2-81d3d3d335cb service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:46.969068 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 828/1646] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:51:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:51:47.028511 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1312a4c-2266-4098-b71d-3aa7b9929747 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:47.030063 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1312a4c-2266-4098-b71d-3aa7b9929747 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:47.061800 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 819/1647] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:51:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:51:47.622912 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a9322-bf15-4160-a1d5-e7fe237bb6f0 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:47.623203 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a9322-bf15-4160-a1d5-e7fe237bb6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:47.623501 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a9322-bf15-4160-a1d5-e7fe237bb6f0 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:47.625851 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 829/1648] 10.222.0.97 () {54 vars in 962 bytes} [Tue Jul 26 14:51:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:51:47.648203 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453bf14-099e-4bcd-8457-47aaafe1570a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:47.648868 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453bf14-099e-4bcd-8457-47aaafe1570a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:47.649336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453bf14-099e-4bcd-8457-47aaafe1570a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:47.692577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3453bf14-099e-4bcd-8457-47aaafe1570a None None] MFA Rules not processed for user `266abef6cda5495b897170f92043aa03`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:47.694173 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3453bf14-099e-4bcd-8457-47aaafe1570a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:47.739342 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:51:47.840378 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 820/1649] 10.222.0.97 () {58 vars in 1059 bytes} [Tue Jul 26 14:51:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:47.847671 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac4b8d-f78d-4fe9-83f5-8d60e0b2039e None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:47.848332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac4b8d-f78d-4fe9-83f5-8d60e0b2039e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:47.848535 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac4b8d-f78d-4fe9-83f5-8d60e0b2039e None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:47.850516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 830/1650] 10.222.0.97 () {54 vars in 912 bytes} [Tue Jul 26 14:51:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:51:47.869673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411cd222-8c51-4e50-815d-3a64983b0cce None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:47.870324 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411cd222-8c51-4e50-815d-3a64983b0cce None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:47.870968 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411cd222-8c51-4e50-815d-3a64983b0cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:47.912691 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-411cd222-8c51-4e50-815d-3a64983b0cce None None] MFA Rules not processed for user `266abef6cda5495b897170f92043aa03`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:47.914912 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-411cd222-8c51-4e50-815d-3a64983b0cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:47.945333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 821/1651] 10.222.0.97 () {58 vars in 1009 bytes} [Tue Jul 26 14:51:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:47.958574 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:51:48.017633 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac4b8d-f78d-4fe9-83f5-8d60e0b2039e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:48.030904 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac4b8d-f78d-4fe9-83f5-8d60e0b2039e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:48.036464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0098afe8-ddae-4a5d-95bc-e92dde9d4701 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:48.037265 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098afe8-ddae-4a5d-95bc-e92dde9d4701 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:48.037752 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098afe8-ddae-4a5d-95bc-e92dde9d4701 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:48.038239 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098afe8-ddae-4a5d-95bc-e92dde9d4701 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:48.059343 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0098afe8-ddae-4a5d-95bc-e92dde9d4701 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:48.061334 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0098afe8-ddae-4a5d-95bc-e92dde9d4701 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:48.079749 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 831/1652] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:51:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:51:49.093816 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411cd222-8c51-4e50-815d-3a64983b0cce None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:49.382312 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411cd222-8c51-4e50-815d-3a64983b0cce None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:49.400549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a25ca1-6215-4651-9a02-93da0359e823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:49.401253 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a25ca1-6215-4651-9a02-93da0359e823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:49.401540 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a25ca1-6215-4651-9a02-93da0359e823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:49.401822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a25ca1-6215-4651-9a02-93da0359e823 admin admin] PATH_INFO: `/v3/users/266b64f12ac44348aaa2742a16dd4b1e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:49.404024 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a25ca1-6215-4651-9a02-93da0359e823 admin admin] RBAC: Authorizing `identity:delete_user(user_id=266b64f12ac44348aaa2742a16dd4b1e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:49.405207 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: WARNING py.warnings [None req-c7a25ca1-6215-4651-9a02-93da0359e823 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 14:51:49.405207 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: warnings.warn(msg) Jul 26 14:51:49.405207 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: Jul 26 14:51:49.406634 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a25ca1-6215-4651-9a02-93da0359e823 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:49.914030 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c7a25ca1-6215-4651-9a02-93da0359e823 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '266b64f12ac44348aaa2742a16dd4b1e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:51:49.915986 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c7a25ca1-6215-4651-9a02-93da0359e823 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '266b64f12ac44348aaa2742a16dd4b1e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:51:49.987653 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c7a25ca1-6215-4651-9a02-93da0359e823 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '266b64f12ac44348aaa2742a16dd4b1e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:51:50.026101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c7a25ca1-6215-4651-9a02-93da0359e823 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '266b64f12ac44348aaa2742a16dd4b1e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:51:50.030840 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c7a25ca1-6215-4651-9a02-93da0359e823 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '266b64f12ac44348aaa2742a16dd4b1e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:51:50.190504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 822/1653] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:51:49 2022] DELETE /identity/v3/users/266b64f12ac44348aaa2742a16dd4b1e => generated 0 bytes in 1096 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:51:50.194989 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6aed3b-f671-454d-8ef2-e71d767da6de None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:50.195539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6aed3b-f671-454d-8ef2-e71d767da6de None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:50.195608 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6aed3b-f671-454d-8ef2-e71d767da6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:50.227504 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-bb6aed3b-f671-454d-8ef2-e71d767da6de None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:50.232803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-bb6aed3b-f671-454d-8ef2-e71d767da6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:50.246133 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:51:50.332386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:51:50.348364 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 832/1654] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:50.367105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a25ca1-6215-4651-9a02-93da0359e823 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:50.368907 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c7a25ca1-6215-4651-9a02-93da0359e823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:50.479292 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a25ca1-6215-4651-9a02-93da0359e823 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:50.484754 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df990bd3-6e1f-44cb-9318-8dd7f4fdba98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:50.485715 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df990bd3-6e1f-44cb-9318-8dd7f4fdba98 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:50.486184 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df990bd3-6e1f-44cb-9318-8dd7f4fdba98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:50.486637 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df990bd3-6e1f-44cb-9318-8dd7f4fdba98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:50.512217 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df990bd3-6e1f-44cb-9318-8dd7f4fdba98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:50.513911 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df990bd3-6e1f-44cb-9318-8dd7f4fdba98 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:50.555273 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 823/1655] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:51:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:51:52.363746 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6aed3b-f671-454d-8ef2-e71d767da6de None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:52.365217 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-bb6aed3b-f671-454d-8ef2-e71d767da6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:52.386487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6aed3b-f671-454d-8ef2-e71d767da6de None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:52.396687 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab57537e-0b09-433d-8425-c44c13ad5ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:52.405572 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57537e-0b09-433d-8425-c44c13ad5ab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:52.406142 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57537e-0b09-433d-8425-c44c13ad5ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:52.406585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57537e-0b09-433d-8425-c44c13ad5ab9 admin admin] PATH_INFO: `/v3/projects/2f0022e5d6334556bca1b43bce6f6ecf` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:52.414066 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab57537e-0b09-433d-8425-c44c13ad5ab9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f0022e5d6334556bca1b43bce6f6ecf)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:52.415576 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: WARNING py.warnings [None req-ab57537e-0b09-433d-8425-c44c13ad5ab9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 14:51:52.415576 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: warnings.warn(msg) Jul 26 14:51:52.415576 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: Jul 26 14:51:52.417358 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab57537e-0b09-433d-8425-c44c13ad5ab9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:52.683344 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ab57537e-0b09-433d-8425-c44c13ad5ab9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f0022e5d6334556bca1b43bce6f6ecf', 'request_id': 'req-ab57537e-0b09-433d-8425-c44c13ad5ab9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:51:52.710819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ab57537e-0b09-433d-8425-c44c13ad5ab9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f0022e5d6334556bca1b43bce6f6ecf', 'request_id': 'req-ab57537e-0b09-433d-8425-c44c13ad5ab9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:51:52.827550 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ab57537e-0b09-433d-8425-c44c13ad5ab9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f0022e5d6334556bca1b43bce6f6ecf', 'request_id': 'req-ab57537e-0b09-433d-8425-c44c13ad5ab9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:51:52.874714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305be236-4344-437e-8cb3-4e3eb33ef740 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:52.875241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305be236-4344-437e-8cb3-4e3eb33ef740 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:52.875308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305be236-4344-437e-8cb3-4e3eb33ef740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:52.882320 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ab57537e-0b09-433d-8425-c44c13ad5ab9 admin admin] The token cache is being invalidate because project 2f0022e5d6334556bca1b43bce6f6ecf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:51:52.882645 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ab57537e-0b09-433d-8425-c44c13ad5ab9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:51:52.886626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ab57537e-0b09-433d-8425-c44c13ad5ab9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:51:52.889740 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 833/1656] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:51:52 2022] DELETE /identity/v3/projects/2f0022e5d6334556bca1b43bce6f6ecf => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:51:52.894919 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab57537e-0b09-433d-8425-c44c13ad5ab9 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:52.904054 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ab57537e-0b09-433d-8425-c44c13ad5ab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:52.953692 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-305be236-4344-437e-8cb3-4e3eb33ef740 None None] MFA Rules not processed for user `de3b593763e44e28be0f51187defa75d`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:52.955218 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-305be236-4344-437e-8cb3-4e3eb33ef740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:52.981143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:51:53.100377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 824/1657] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:51:53.104210 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab57537e-0b09-433d-8425-c44c13ad5ab9 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:53.105398 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305be236-4344-437e-8cb3-4e3eb33ef740 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:53.114083 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-305be236-4344-437e-8cb3-4e3eb33ef740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:53.125177 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6cfc94-7d35-4d7b-b1d0-6296c5be8d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:53.126960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6cfc94-7d35-4d7b-b1d0-6296c5be8d29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:53.128260 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6cfc94-7d35-4d7b-b1d0-6296c5be8d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:53.130356 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6cfc94-7d35-4d7b-b1d0-6296c5be8d29 admin admin] PATH_INFO: `/v3/users/73e4210496a14e1ba33768dec4a128eb` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:53.133149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6cfc94-7d35-4d7b-b1d0-6296c5be8d29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73e4210496a14e1ba33768dec4a128eb)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:53.148711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6cfc94-7d35-4d7b-b1d0-6296c5be8d29 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:53.264751 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305be236-4344-437e-8cb3-4e3eb33ef740 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:53.274910 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1778db0a-f4a9-42b4-9fdc-ac1907ec1cd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:53.276133 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778db0a-f4a9-42b4-9fdc-ac1907ec1cd5 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:53.276421 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778db0a-f4a9-42b4-9fdc-ac1907ec1cd5 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:53.277259 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778db0a-f4a9-42b4-9fdc-ac1907ec1cd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:53.288128 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1778db0a-f4a9-42b4-9fdc-ac1907ec1cd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:53.290065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1778db0a-f4a9-42b4-9fdc-ac1907ec1cd5 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:53.302104 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 825/1658] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:51:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:51:53.479922 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-aa6cfc94-7d35-4d7b-b1d0-6296c5be8d29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73e4210496a14e1ba33768dec4a128eb'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:51:53.481332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-aa6cfc94-7d35-4d7b-b1d0-6296c5be8d29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73e4210496a14e1ba33768dec4a128eb'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:51:53.504494 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-aa6cfc94-7d35-4d7b-b1d0-6296c5be8d29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73e4210496a14e1ba33768dec4a128eb'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:51:53.517031 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-aa6cfc94-7d35-4d7b-b1d0-6296c5be8d29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73e4210496a14e1ba33768dec4a128eb'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:51:53.518822 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-aa6cfc94-7d35-4d7b-b1d0-6296c5be8d29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73e4210496a14e1ba33768dec4a128eb'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:51:53.642339 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:51:53.663039 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 834/1659] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:51:52 2022] DELETE /identity/v3/users/73e4210496a14e1ba33768dec4a128eb => generated 0 bytes in 769 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:51:54.927653 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1778db0a-f4a9-42b4-9fdc-ac1907ec1cd5 service nova] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:54.945212 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1778db0a-f4a9-42b4-9fdc-ac1907ec1cd5 service nova] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:55.017299 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6cfc94-7d35-4d7b-b1d0-6296c5be8d29 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:55.022109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-aa6cfc94-7d35-4d7b-b1d0-6296c5be8d29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:55.141774 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab907d-1064-42fd-ad44-931c3bbc021d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:55.142456 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab907d-1064-42fd-ad44-931c3bbc021d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:55.142742 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab907d-1064-42fd-ad44-931c3bbc021d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:55.143023 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab907d-1064-42fd-ad44-931c3bbc021d admin admin] PATH_INFO: `/v3/projects/d6e4536022c94225b14eba5d808bc436` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:55.146891 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ab907d-1064-42fd-ad44-931c3bbc021d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6e4536022c94225b14eba5d808bc436)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:55.148915 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ab907d-1064-42fd-ad44-931c3bbc021d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:55.278476 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6cfc94-7d35-4d7b-b1d0-6296c5be8d29 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:55.297201 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33a884-3ddb-46a4-967b-29a69a7fa30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:55.302829 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33a884-3ddb-46a4-967b-29a69a7fa30a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:55.303503 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33a884-3ddb-46a4-967b-29a69a7fa30a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:55.307154 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33a884-3ddb-46a4-967b-29a69a7fa30a admin admin] PATH_INFO: `/v3/users/904ae63d554f4435ad5f7702bea2d3ba` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:55.311513 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33a884-3ddb-46a4-967b-29a69a7fa30a admin admin] RBAC: Authorizing `identity:delete_user(user_id=904ae63d554f4435ad5f7702bea2d3ba)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:55.314256 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33a884-3ddb-46a4-967b-29a69a7fa30a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:55.349914 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-99ab907d-1064-42fd-ad44-931c3bbc021d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6e4536022c94225b14eba5d808bc436', 'request_id': 'req-99ab907d-1064-42fd-ad44-931c3bbc021d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:51:55.357972 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-99ab907d-1064-42fd-ad44-931c3bbc021d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6e4536022c94225b14eba5d808bc436', 'request_id': 'req-99ab907d-1064-42fd-ad44-931c3bbc021d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:51:55.450830 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-99ab907d-1064-42fd-ad44-931c3bbc021d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6e4536022c94225b14eba5d808bc436', 'request_id': 'req-99ab907d-1064-42fd-ad44-931c3bbc021d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:51:55.470880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-99ab907d-1064-42fd-ad44-931c3bbc021d admin admin] The token cache is being invalidate because project d6e4536022c94225b14eba5d808bc436 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:51:55.471386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-99ab907d-1064-42fd-ad44-931c3bbc021d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:51:55.473590 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-99ab907d-1064-42fd-ad44-931c3bbc021d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:51:55.478885 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 826/1660] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:51:54 2022] DELETE /identity/v3/projects/d6e4536022c94225b14eba5d808bc436 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:51:55.483792 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b398fe-0574-4132-9ca2-33f3d16b89fd None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:55.484228 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b398fe-0574-4132-9ca2-33f3d16b89fd None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:55.485932 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b398fe-0574-4132-9ca2-33f3d16b89fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:55.518398 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-45b398fe-0574-4132-9ca2-33f3d16b89fd None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:55.521356 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-45b398fe-0574-4132-9ca2-33f3d16b89fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:55.636319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4a33a884-3ddb-46a4-967b-29a69a7fa30a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '904ae63d554f4435ad5f7702bea2d3ba'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:51:55.638095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4a33a884-3ddb-46a4-967b-29a69a7fa30a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '904ae63d554f4435ad5f7702bea2d3ba'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:51:55.669557 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4a33a884-3ddb-46a4-967b-29a69a7fa30a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '904ae63d554f4435ad5f7702bea2d3ba'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:51:55.689157 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4a33a884-3ddb-46a4-967b-29a69a7fa30a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '904ae63d554f4435ad5f7702bea2d3ba'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:51:55.693215 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4a33a884-3ddb-46a4-967b-29a69a7fa30a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '904ae63d554f4435ad5f7702bea2d3ba'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:51:55.718227 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 827/1661] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:55.727341 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef028cd0-c68f-4850-8c76-46951a7dfd63 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:55.727801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef028cd0-c68f-4850-8c76-46951a7dfd63 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:55.728189 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef028cd0-c68f-4850-8c76-46951a7dfd63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:55.771475 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ef028cd0-c68f-4850-8c76-46951a7dfd63 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:55.773426 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ef028cd0-c68f-4850-8c76-46951a7dfd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:55.796019 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 828/1662] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:55.800346 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef028cd0-c68f-4850-8c76-46951a7dfd63 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:55.818484 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 835/1663] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:51:55 2022] DELETE /identity/v3/users/904ae63d554f4435ad5f7702bea2d3ba => generated 0 bytes in 802 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:51:55.825767 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b84f42-37c0-47e0-94f7-d2f4914f28e9 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:55.826471 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b84f42-37c0-47e0-94f7-d2f4914f28e9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:55.826948 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b84f42-37c0-47e0-94f7-d2f4914f28e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:55.829959 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef028cd0-c68f-4850-8c76-46951a7dfd63 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:55.876623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-55b84f42-37c0-47e0-94f7-d2f4914f28e9 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:55.880927 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-55b84f42-37c0-47e0-94f7-d2f4914f28e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:55.995985 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab9234-9f6a-448a-97be-67d892253d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:55.996785 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab9234-9f6a-448a-97be-67d892253d9d admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:55.997229 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab9234-9f6a-448a-97be-67d892253d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:55.997650 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab9234-9f6a-448a-97be-67d892253d9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:55.998593 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab9234-9f6a-448a-97be-67d892253d9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:51:55.999773 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab9234-9f6a-448a-97be-67d892253d9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:56.002068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab9234-9f6a-448a-97be-67d892253d9d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:56.027193 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:51:56.055491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 836/1664] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:51:56.055979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 829/1665] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:51:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:51:56.056589 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b84f42-37c0-47e0-94f7-d2f4914f28e9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:56.071382 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b84f42-37c0-47e0-94f7-d2f4914f28e9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:56.100043 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef20157-20eb-43af-9f10-63a2da0aa4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:56.101214 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab9234-9f6a-448a-97be-67d892253d9d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:56.103330 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8aab9234-9f6a-448a-97be-67d892253d9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:56.103596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef20157-20eb-43af-9f10-63a2da0aa4c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:56.104291 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef20157-20eb-43af-9f10-63a2da0aa4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:56.105917 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef20157-20eb-43af-9f10-63a2da0aa4c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:56.107754 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef20157-20eb-43af-9f10-63a2da0aa4c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:56.113990 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef20157-20eb-43af-9f10-63a2da0aa4c9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:56.220411 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab9234-9f6a-448a-97be-67d892253d9d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:56.233729 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46850468-331e-41ff-ab41-fb59b8ded147 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:56.234606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46850468-331e-41ff-ab41-fb59b8ded147 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:56.235025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46850468-331e-41ff-ab41-fb59b8ded147 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:56.235501 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46850468-331e-41ff-ab41-fb59b8ded147 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:56.245920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 837/1666] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:51:56 2022] POST /identity/v3/projects => generated 385 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:51:56.255565 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef20157-20eb-43af-9f10-63a2da0aa4c9 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:56.264003 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46850468-331e-41ff-ab41-fb59b8ded147 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:56.265473 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46850468-331e-41ff-ab41-fb59b8ded147 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:56.289174 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 830/1667] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:51:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:51:56.445260 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef20157-20eb-43af-9f10-63a2da0aa4c9 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:56.458990 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b28b20c-071d-42e4-bab3-34626a0ec2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:56.459676 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28b20c-071d-42e4-bab3-34626a0ec2fb admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:56.459974 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28b20c-071d-42e4-bab3-34626a0ec2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:56.460234 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28b20c-071d-42e4-bab3-34626a0ec2fb admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:56.461516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b28b20c-071d-42e4-bab3-34626a0ec2fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:56.463140 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b28b20c-071d-42e4-bab3-34626a0ec2fb admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:56.813040 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46850468-331e-41ff-ab41-fb59b8ded147 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:56.813163 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 838/1668] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:51:56 2022] POST /identity/v3/users => generated 293 bytes in 557 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:51:56.832705 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46850468-331e-41ff-ab41-fb59b8ded147 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:56.862238 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c73a35-c338-4511-887d-0f96ac4f01a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:56.863535 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c73a35-c338-4511-887d-0f96ac4f01a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:56.863822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c73a35-c338-4511-887d-0f96ac4f01a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:56.864096 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c73a35-c338-4511-887d-0f96ac4f01a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:56.864802 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c73a35-c338-4511-887d-0f96ac4f01a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:56.868465 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c73a35-c338-4511-887d-0f96ac4f01a3 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:56.923559 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 831/1669] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:51:56.927766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b28b20c-071d-42e4-bab3-34626a0ec2fb admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:56.948033 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b28b20c-071d-42e4-bab3-34626a0ec2fb admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:56.986824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de98f0-8817-45b2-b23c-89ad0e392dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:56.988249 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de98f0-8817-45b2-b23c-89ad0e392dca admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:56.988679 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de98f0-8817-45b2-b23c-89ad0e392dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:56.989094 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de98f0-8817-45b2-b23c-89ad0e392dca admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:56.989968 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70de98f0-8817-45b2-b23c-89ad0e392dca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:56.991634 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70de98f0-8817-45b2-b23c-89ad0e392dca admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:57.027041 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 839/1670] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:51:57.033892 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c73a35-c338-4511-887d-0f96ac4f01a3 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:57.064492 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c73a35-c338-4511-887d-0f96ac4f01a3 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:57.089803 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73be8ee-3ade-45bc-bdf0-6de1ccdcb162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:57.090518 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73be8ee-3ade-45bc-bdf0-6de1ccdcb162 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:57.091109 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73be8ee-3ade-45bc-bdf0-6de1ccdcb162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:57.091560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73be8ee-3ade-45bc-bdf0-6de1ccdcb162 admin admin] PATH_INFO: `/v3/projects/1f77b8972e364573ba62facb721fa7bd/users/670579194ff445bb93adfd71376e66a4/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:57.123715 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73be8ee-3ade-45bc-bdf0-6de1ccdcb162 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f77b8972e364573ba62facb721fa7bd, user_id=670579194ff445bb93adfd71376e66a4, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:57.126662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73be8ee-3ade-45bc-bdf0-6de1ccdcb162 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:57.185906 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:51:57.259151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 832/1671] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:51:57 2022] PUT /identity/v3/projects/1f77b8972e364573ba62facb721fa7bd/users/670579194ff445bb93adfd71376e66a4/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:51:57.269654 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478d0f8-c6bd-423e-9982-f6240f1dc252 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:57.269945 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478d0f8-c6bd-423e-9982-f6240f1dc252 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:57.270227 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478d0f8-c6bd-423e-9982-f6240f1dc252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:57.354108 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a478d0f8-c6bd-423e-9982-f6240f1dc252 None None] MFA Rules not processed for user `670579194ff445bb93adfd71376e66a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:57.358080 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a478d0f8-c6bd-423e-9982-f6240f1dc252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:57.487889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 840/1672] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:57.492736 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73be8ee-3ade-45bc-bdf0-6de1ccdcb162 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:57.632834 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73be8ee-3ade-45bc-bdf0-6de1ccdcb162 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:57.660741 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf7cf6-709a-4d72-b5d6-36c4053f3305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:57.661383 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf7cf6-709a-4d72-b5d6-36c4053f3305 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:57.661669 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf7cf6-709a-4d72-b5d6-36c4053f3305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:57.661946 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf7cf6-709a-4d72-b5d6-36c4053f3305 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:57.662627 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bf7cf6-709a-4d72-b5d6-36c4053f3305 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:57.664599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bf7cf6-709a-4d72-b5d6-36c4053f3305 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:57.851173 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 833/1673] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:51:57 2022] POST /identity/v3/projects => generated 387 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:51:57.855491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a478d0f8-c6bd-423e-9982-f6240f1dc252 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:57.874422 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a478d0f8-c6bd-423e-9982-f6240f1dc252 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:57.906458 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45dd6c6-326c-4122-a82e-91ef7646b984 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:57.907068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45dd6c6-326c-4122-a82e-91ef7646b984 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:57.907581 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45dd6c6-326c-4122-a82e-91ef7646b984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:57.964969 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a45dd6c6-326c-4122-a82e-91ef7646b984 None None] MFA Rules not processed for user `e0615d20057342cfbd16aa0ec94d1979`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:57.966780 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a45dd6c6-326c-4122-a82e-91ef7646b984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:58.030533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f1258-0b43-4f7f-bf76-c6b03faaff14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:58.031446 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f1258-0b43-4f7f-bf76-c6b03faaff14 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:58.031894 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f1258-0b43-4f7f-bf76-c6b03faaff14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:58.037530 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f1258-0b43-4f7f-bf76-c6b03faaff14 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:58.037530 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f1258-0b43-4f7f-bf76-c6b03faaff14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:58.039847 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f1258-0b43-4f7f-bf76-c6b03faaff14 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:58.140719 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 834/1674] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:51:58.140719 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45dd6c6-326c-4122-a82e-91ef7646b984 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:58.140719 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a45dd6c6-326c-4122-a82e-91ef7646b984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:58.281100 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 841/1675] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:51:57 2022] POST /identity/v3/users => generated 300 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:51:58.286005 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f1258-0b43-4f7f-bf76-c6b03faaff14 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:58.319917 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f1258-0b43-4f7f-bf76-c6b03faaff14 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:58.353002 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666ffe1-85d7-4404-b2a4-53d3ccf02080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:58.353640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666ffe1-85d7-4404-b2a4-53d3ccf02080 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:58.358312 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666ffe1-85d7-4404-b2a4-53d3ccf02080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:58.358598 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666ffe1-85d7-4404-b2a4-53d3ccf02080 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:58.359352 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666ffe1-85d7-4404-b2a4-53d3ccf02080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:58.372666 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666ffe1-85d7-4404-b2a4-53d3ccf02080 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:58.398931 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 842/1676] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:51:58.400179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666ffe1-85d7-4404-b2a4-53d3ccf02080 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:58.417739 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2666ffe1-85d7-4404-b2a4-53d3ccf02080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:58.421223 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:51:58.450560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45dd6c6-326c-4122-a82e-91ef7646b984 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:58.472056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666ffe1-85d7-4404-b2a4-53d3ccf02080 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:58.478911 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a4f89-3e12-4c8e-9a17-910b880154e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:58.486917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a4f89-3e12-4c8e-9a17-910b880154e8 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:58.487473 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a4f89-3e12-4c8e-9a17-910b880154e8 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:58.488415 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a4f89-3e12-4c8e-9a17-910b880154e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:58.505207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5497258a-aa03-4bf6-a41b-e029cb5ba0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:58.507169 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5497258a-aa03-4bf6-a41b-e029cb5ba0af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:58.507497 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5497258a-aa03-4bf6-a41b-e029cb5ba0af admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:58.507786 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5497258a-aa03-4bf6-a41b-e029cb5ba0af admin admin] PATH_INFO: `/v3/projects/6a4d0de164894881bcec30ff989a283b` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:58.512352 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5497258a-aa03-4bf6-a41b-e029cb5ba0af admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a4d0de164894881bcec30ff989a283b)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:58.518106 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a4f89-3e12-4c8e-9a17-910b880154e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:58.518325 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5497258a-aa03-4bf6-a41b-e029cb5ba0af admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:58.522930 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a4f89-3e12-4c8e-9a17-910b880154e8 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:58.556564 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 835/1677] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:51:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:51:58.557707 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a4f89-3e12-4c8e-9a17-910b880154e8 service nova] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:58.608043 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a4f89-3e12-4c8e-9a17-910b880154e8 service nova] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:58.638102 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d21c2-ed1c-4803-b121-250253c3b6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:58.638995 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d21c2-ed1c-4803-b121-250253c3b6ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:58.639095 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d21c2-ed1c-4803-b121-250253c3b6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:58.639360 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d21c2-ed1c-4803-b121-250253c3b6ba admin admin] PATH_INFO: `/v3/projects/52f16ee6af45442ca660a09f20cf5b89/users/0026e58f974d419b8cecf878ab00a38c/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:58.703010 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d21c2-ed1c-4803-b121-250253c3b6ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=52f16ee6af45442ca660a09f20cf5b89, user_id=0026e58f974d419b8cecf878ab00a38c, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:58.709868 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d21c2-ed1c-4803-b121-250253c3b6ba admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:58.873894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 836/1678] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:51:58 2022] PUT /identity/v3/projects/52f16ee6af45442ca660a09f20cf5b89/users/0026e58f974d419b8cecf878ab00a38c/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:51:58.877286 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b823f62-7825-4ea4-96a4-38be6e611080 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:58.881583 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b823f62-7825-4ea4-96a4-38be6e611080 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:58.882259 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b823f62-7825-4ea4-96a4-38be6e611080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:58.895905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5497258a-aa03-4bf6-a41b-e029cb5ba0af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a4d0de164894881bcec30ff989a283b', 'request_id': 'req-5497258a-aa03-4bf6-a41b-e029cb5ba0af'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:51:58.906079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5497258a-aa03-4bf6-a41b-e029cb5ba0af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a4d0de164894881bcec30ff989a283b', 'request_id': 'req-5497258a-aa03-4bf6-a41b-e029cb5ba0af'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:51:58.933466 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-7b823f62-7825-4ea4-96a4-38be6e611080 None None] MFA Rules not processed for user `713d2a0f610d4296887b6ab9664ae5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:58.934711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7b823f62-7825-4ea4-96a4-38be6e611080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:58.999958 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5497258a-aa03-4bf6-a41b-e029cb5ba0af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a4d0de164894881bcec30ff989a283b', 'request_id': 'req-5497258a-aa03-4bf6-a41b-e029cb5ba0af'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:51:59.023515 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 837/1679] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:59.023515 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b823f62-7825-4ea4-96a4-38be6e611080 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:59.023846 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5497258a-aa03-4bf6-a41b-e029cb5ba0af admin admin] The token cache is being invalidate because project 6a4d0de164894881bcec30ff989a283b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:51:59.024585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5497258a-aa03-4bf6-a41b-e029cb5ba0af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:51:59.025533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5497258a-aa03-4bf6-a41b-e029cb5ba0af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:51:59.030796 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 843/1680] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:51:58 2022] DELETE /identity/v3/projects/6a4d0de164894881bcec30ff989a283b => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:51:59.031392 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f166fef-8270-42d3-9dc2-0d01f184bced None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:59.031744 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f166fef-8270-42d3-9dc2-0d01f184bced None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:59.032078 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f166fef-8270-42d3-9dc2-0d01f184bced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:59.091955 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:51:59.096805 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-2f166fef-8270-42d3-9dc2-0d01f184bced None None] MFA Rules not processed for user `19c0408b4321495eabd4738bbb8b711b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:59.098786 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2f166fef-8270-42d3-9dc2-0d01f184bced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:59.103218 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b823f62-7825-4ea4-96a4-38be6e611080 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:59.109516 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba101ca-9425-4eb9-be1f-c40f82d87ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:59.110168 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba101ca-9425-4eb9-be1f-c40f82d87ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:59.110455 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba101ca-9425-4eb9-be1f-c40f82d87ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:59.110738 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba101ca-9425-4eb9-be1f-c40f82d87ce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:59.111959 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba101ca-9425-4eb9-be1f-c40f82d87ce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:59.113628 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba101ca-9425-4eb9-be1f-c40f82d87ce9 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:59.126995 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 838/1681] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:51:59.129525 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba868a55-cc8b-4bd7-a6ff-90e85060518e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:59.129964 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba868a55-cc8b-4bd7-a6ff-90e85060518e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:59.130881 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba868a55-cc8b-4bd7-a6ff-90e85060518e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:59.159385 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ba868a55-cc8b-4bd7-a6ff-90e85060518e None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:59.163786 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ba868a55-cc8b-4bd7-a6ff-90e85060518e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:59.186473 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 839/1682] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:59.186473 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59061335-4db0-4fb4-97b7-5a63b93fe433 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:59.186473 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59061335-4db0-4fb4-97b7-5a63b93fe433 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:59.186473 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59061335-4db0-4fb4-97b7-5a63b93fe433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:59.208415 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-59061335-4db0-4fb4-97b7-5a63b93fe433 None None] MFA Rules not processed for user `713d2a0f610d4296887b6ab9664ae5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:59.209649 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-59061335-4db0-4fb4-97b7-5a63b93fe433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:59.215555 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 844/1683] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:59.216445 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f166fef-8270-42d3-9dc2-0d01f184bced None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:59.223452 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2f166fef-8270-42d3-9dc2-0d01f184bced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:59.229175 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 840/1684] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:59.229175 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a35385-f8c3-4835-ace6-2cf918b08fbd None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:59.229175 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a35385-f8c3-4835-ace6-2cf918b08fbd None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:59.229175 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a35385-f8c3-4835-ace6-2cf918b08fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:59.247242 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f166fef-8270-42d3-9dc2-0d01f184bced None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:59.252008 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e55047a-364f-4693-a525-07f6b71c03f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:59.252163 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e4a35385-f8c3-4835-ace6-2cf918b08fbd None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:59.252706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55047a-364f-4693-a525-07f6b71c03f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:59.252993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55047a-364f-4693-a525-07f6b71c03f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:59.253261 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e4a35385-f8c3-4835-ace6-2cf918b08fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:59.253352 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55047a-364f-4693-a525-07f6b71c03f6 admin admin] PATH_INFO: `/v3/projects/52f16ee6af45442ca660a09f20cf5b89/users/0026e58f974d419b8cecf878ab00a38c/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:59.255687 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e55047a-364f-4693-a525-07f6b71c03f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52f16ee6af45442ca660a09f20cf5b89, user_id=0026e58f974d419b8cecf878ab00a38c, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:59.258376 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e55047a-364f-4693-a525-07f6b71c03f6 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:59.261691 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 841/1685] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:59.263275 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03d310-1322-4153-93fa-10d1bd9f5377 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:59.263630 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03d310-1322-4153-93fa-10d1bd9f5377 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:59.263911 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03d310-1322-4153-93fa-10d1bd9f5377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:59.277152 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 845/1686] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:51:59 2022] PUT /identity/v3/projects/52f16ee6af45442ca660a09f20cf5b89/users/0026e58f974d419b8cecf878ab00a38c/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:51:59.278307 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e55047a-364f-4693-a525-07f6b71c03f6 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:59.281729 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8e55047a-364f-4693-a525-07f6b71c03f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:59.301230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6e03d310-1322-4153-93fa-10d1bd9f5377 None None] MFA Rules not processed for user `19c0408b4321495eabd4738bbb8b711b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:59.301669 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6e03d310-1322-4153-93fa-10d1bd9f5377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:59.419561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e55047a-364f-4693-a525-07f6b71c03f6 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:59.435841 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06f6d6-019f-42b9-be5b-7eac001b52d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:59.437645 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06f6d6-019f-42b9-be5b-7eac001b52d4 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:59.438398 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06f6d6-019f-42b9-be5b-7eac001b52d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:59.444768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06f6d6-019f-42b9-be5b-7eac001b52d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:59.444934 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 842/1687] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:59.444934 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03d310-1322-4153-93fa-10d1bd9f5377 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:59.446913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3d06f6d6-019f-42b9-be5b-7eac001b52d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:59.559728 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03d310-1322-4153-93fa-10d1bd9f5377 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:59.566382 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af05f63d-3e57-405b-a01f-7e38c2096b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:59.567114 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05f63d-3e57-405b-a01f-7e38c2096b09 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:59.567422 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05f63d-3e57-405b-a01f-7e38c2096b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:59.567753 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05f63d-3e57-405b-a01f-7e38c2096b09 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:59.569431 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af05f63d-3e57-405b-a01f-7e38c2096b09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:51:59.570122 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af05f63d-3e57-405b-a01f-7e38c2096b09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:59.573047 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d06f6d6-019f-42b9-be5b-7eac001b52d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:59.574127 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af05f63d-3e57-405b-a01f-7e38c2096b09 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:59.579989 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d06f6d6-019f-42b9-be5b-7eac001b52d4 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:59.591174 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 843/1688] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:51:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:51:59.593219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5b715-c794-47c6-8546-18e25b350f66 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:59.593455 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5b715-c794-47c6-8546-18e25b350f66 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:59.593740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5b715-c794-47c6-8546-18e25b350f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:59.600986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 846/1689] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:51:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:51:59.602007 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06f6d6-019f-42b9-be5b-7eac001b52d4 service nova] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:59.606089 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3d06f6d6-019f-42b9-be5b-7eac001b52d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:59.640703 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-25c5b715-c794-47c6-8546-18e25b350f66 None None] MFA Rules not processed for user `0026e58f974d419b8cecf878ab00a38c`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:51:59.644838 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-25c5b715-c794-47c6-8546-18e25b350f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:59.646035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06f6d6-019f-42b9-be5b-7eac001b52d4 service nova] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:59.671880 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f08044-a3ce-4f0b-8f13-b0d1b06f04a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:59.672654 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f08044-a3ce-4f0b-8f13-b0d1b06f04a9 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:59.672942 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f08044-a3ce-4f0b-8f13-b0d1b06f04a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:59.673219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f08044-a3ce-4f0b-8f13-b0d1b06f04a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:59.696234 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f08044-a3ce-4f0b-8f13-b0d1b06f04a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:59.700659 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f08044-a3ce-4f0b-8f13-b0d1b06f04a9 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:59.723882 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 847/1690] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:51:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:51:59.725118 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f08044-a3ce-4f0b-8f13-b0d1b06f04a9 service nova] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:59.742198 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f08044-a3ce-4f0b-8f13-b0d1b06f04a9 service nova] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:51:59.754104 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f1a54-9574-42f1-acfd-d76beb1c9052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:51:59.754744 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f1a54-9574-42f1-acfd-d76beb1c9052 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:51:59.761900 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f1a54-9574-42f1-acfd-d76beb1c9052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:51:59.762283 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f1a54-9574-42f1-acfd-d76beb1c9052 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:51:59.763078 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823f1a54-9574-42f1-acfd-d76beb1c9052 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:51:59.765324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823f1a54-9574-42f1-acfd-d76beb1c9052 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:51:59.798912 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 844/1691] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:51:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:51:59.812757 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c5b715-c794-47c6-8546-18e25b350f66 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:59.814525 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-25c5b715-c794-47c6-8546-18e25b350f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:51:59.915016 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 848/1692] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:51:59 2022] POST /identity/v3/projects => generated 367 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:51:59.921687 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f1a54-9574-42f1-acfd-d76beb1c9052 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:51:59.979195 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c5b715-c794-47c6-8546-18e25b350f66 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:00.070823 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f1a54-9574-42f1-acfd-d76beb1c9052 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:00.087824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5936c-0a95-4fd4-820f-2311ed31f59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:00.090673 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5936c-0a95-4fd4-820f-2311ed31f59d admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:00.091583 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5936c-0a95-4fd4-820f-2311ed31f59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:00.094364 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5936c-0a95-4fd4-820f-2311ed31f59d admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:00.095302 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5936c-0a95-4fd4-820f-2311ed31f59d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:00.100490 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5936c-0a95-4fd4-820f-2311ed31f59d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:00.123693 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b2442-0a4a-41ab-b5b5-ff4b439c0113 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:00.124978 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b2442-0a4a-41ab-b5b5-ff4b439c0113 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:00.125440 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b2442-0a4a-41ab-b5b5-ff4b439c0113 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:00.125882 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b2442-0a4a-41ab-b5b5-ff4b439c0113 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:00.156043 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:00.218879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b2442-0a4a-41ab-b5b5-ff4b439c0113 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:00.220539 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b2442-0a4a-41ab-b5b5-ff4b439c0113 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:00.233102 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 845/1693] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:51:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:52:00.239230 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 849/1694] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:51:59 2022] POST /identity/v3/users => generated 284 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:52:00.246894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b2442-0a4a-41ab-b5b5-ff4b439c0113 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:00.262920 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b2442-0a4a-41ab-b5b5-ff4b439c0113 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:00.271131 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e0454-abd8-4b7a-a327-7de402f380d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:00.272126 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e0454-abd8-4b7a-a327-7de402f380d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:00.272555 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e0454-abd8-4b7a-a327-7de402f380d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:00.279755 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e0454-abd8-4b7a-a327-7de402f380d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:00.280885 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856e0454-abd8-4b7a-a327-7de402f380d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:00.282174 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856e0454-abd8-4b7a-a327-7de402f380d6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:00.301261 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 846/1695] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:52:00.303067 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5936c-0a95-4fd4-820f-2311ed31f59d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:00.321750 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5936c-0a95-4fd4-820f-2311ed31f59d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:00.335874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98d355-ef02-4519-af72-80d3a9e73500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:00.337734 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98d355-ef02-4519-af72-80d3a9e73500 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:00.338294 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98d355-ef02-4519-af72-80d3a9e73500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:00.338895 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98d355-ef02-4519-af72-80d3a9e73500 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:00.339904 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98d355-ef02-4519-af72-80d3a9e73500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:00.342244 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98d355-ef02-4519-af72-80d3a9e73500 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:00.366281 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e0454-abd8-4b7a-a327-7de402f380d6 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:00.387333 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 850/1696] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:52:00.387538 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e0454-abd8-4b7a-a327-7de402f380d6 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:00.388832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98d355-ef02-4519-af72-80d3a9e73500 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:00.404767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86456fb8-963a-4af0-a86b-3a648bc8be69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:00.406330 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86456fb8-963a-4af0-a86b-3a648bc8be69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:00.407452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86456fb8-963a-4af0-a86b-3a648bc8be69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:00.410478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86456fb8-963a-4af0-a86b-3a648bc8be69 admin admin] PATH_INFO: `/v3/projects/0dbe90ceb0184024bc85bef3c688eaff/users/e418eb5ef9df406a94997aef7bbc8e02/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:00.414192 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98d355-ef02-4519-af72-80d3a9e73500 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:00.429762 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b6797-0a28-4424-9c9f-9385b9c52694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:00.431389 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b6797-0a28-4424-9c9f-9385b9c52694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:00.432910 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b6797-0a28-4424-9c9f-9385b9c52694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:00.433453 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b6797-0a28-4424-9c9f-9385b9c52694 admin admin] PATH_INFO: `/v3/users/670579194ff445bb93adfd71376e66a4` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:00.444794 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b6797-0a28-4424-9c9f-9385b9c52694 admin admin] RBAC: Authorizing `identity:delete_user(user_id=670579194ff445bb93adfd71376e66a4)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:00.445558 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b6797-0a28-4424-9c9f-9385b9c52694 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:00.469889 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86456fb8-963a-4af0-a86b-3a648bc8be69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dbe90ceb0184024bc85bef3c688eaff, user_id=e418eb5ef9df406a94997aef7bbc8e02, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:00.480376 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86456fb8-963a-4af0-a86b-3a648bc8be69 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:00.515330 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:00.639174 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 847/1697] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:52:00 2022] PUT /identity/v3/projects/0dbe90ceb0184024bc85bef3c688eaff/users/e418eb5ef9df406a94997aef7bbc8e02/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:00.639174 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9dd7d-7113-431c-a183-6e220aa81024 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:00.639174 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9dd7d-7113-431c-a183-6e220aa81024 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:00.639174 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9dd7d-7113-431c-a183-6e220aa81024 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:00.641492 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 848/1698] 10.222.0.97 () {54 vars in 912 bytes} [Tue Jul 26 14:52:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:52:00.643201 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d05bc-81b4-40a8-86d7-75ea1e3b83db None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:00.643634 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d05bc-81b4-40a8-86d7-75ea1e3b83db None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:00.644040 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d05bc-81b4-40a8-86d7-75ea1e3b83db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:00.687186 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-377d05bc-81b4-40a8-86d7-75ea1e3b83db None None] MFA Rules not processed for user `e418eb5ef9df406a94997aef7bbc8e02`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:00.689293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-377d05bc-81b4-40a8-86d7-75ea1e3b83db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:00.793098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-468b6797-0a28-4424-9c9f-9385b9c52694 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '670579194ff445bb93adfd71376e66a4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:00.793870 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-468b6797-0a28-4424-9c9f-9385b9c52694 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '670579194ff445bb93adfd71376e66a4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:00.802372 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 849/1699] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:00 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:00.803466 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6c221-854f-49ba-bcfe-f7ed04db8326 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:00.803762 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6c221-854f-49ba-bcfe-f7ed04db8326 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:00.810569 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6c221-854f-49ba-bcfe-f7ed04db8326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:00.813992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-468b6797-0a28-4424-9c9f-9385b9c52694 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '670579194ff445bb93adfd71376e66a4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:00.822325 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-468b6797-0a28-4424-9c9f-9385b9c52694 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '670579194ff445bb93adfd71376e66a4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:00.823066 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-468b6797-0a28-4424-9c9f-9385b9c52694 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '670579194ff445bb93adfd71376e66a4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:00.848680 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3ce6c221-854f-49ba-bcfe-f7ed04db8326 None None] MFA Rules not processed for user `c81180fb4ff04c1186a21e43fb597a91`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:00.849962 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3ce6c221-854f-49ba-bcfe-f7ed04db8326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:00.921829 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 850/1700] 10.222.0.97 () {58 vars in 1009 bytes} [Tue Jul 26 14:52:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:00.923119 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ffc27-32e0-4c08-8bc4-f9df325d7de9 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:00.923453 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ffc27-32e0-4c08-8bc4-f9df325d7de9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:00.923723 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ffc27-32e0-4c08-8bc4-f9df325d7de9 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:00.925348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 851/1701] 10.222.0.97 () {54 vars in 912 bytes} [Tue Jul 26 14:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:52:00.926269 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ffc27-32e0-4c08-8bc4-f9df325d7de9 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:00.928170 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-232ffc27-32e0-4c08-8bc4-f9df325d7de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:00.983724 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 851/1702] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:52:00 2022] DELETE /identity/v3/users/670579194ff445bb93adfd71376e66a4 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:00.985089 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c0931-c59a-4a0a-901a-53297be3bc51 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:00.985389 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c0931-c59a-4a0a-901a-53297be3bc51 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:00.985670 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c0931-c59a-4a0a-901a-53297be3bc51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:01.031410 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-b15c0931-c59a-4a0a-901a-53297be3bc51 None None] MFA Rules not processed for user `de3b593763e44e28be0f51187defa75d`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:01.034492 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b15c0931-c59a-4a0a-901a-53297be3bc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:01.056293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ffc27-32e0-4c08-8bc4-f9df325d7de9 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:01.145467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38ab9a-69db-408e-aea4-ee4fa814c12c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:01.146121 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38ab9a-69db-408e-aea4-ee4fa814c12c service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:01.146391 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38ab9a-69db-408e-aea4-ee4fa814c12c service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:01.146673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38ab9a-69db-408e-aea4-ee4fa814c12c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:01.148710 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1f38ab9a-69db-408e-aea4-ee4fa814c12c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:01.213263 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 852/1703] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:01.216162 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c81092-4808-4aa2-bbca-73b545e78d12 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:01.217681 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c81092-4808-4aa2-bbca-73b545e78d12 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:01.217973 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c81092-4808-4aa2-bbca-73b545e78d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:01.255521 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a6c81092-4808-4aa2-bbca-73b545e78d12 None None] MFA Rules not processed for user `c81180fb4ff04c1186a21e43fb597a91`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:01.256948 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a6c81092-4808-4aa2-bbca-73b545e78d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:01.260634 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f38ab9a-69db-408e-aea4-ee4fa814c12c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:01.263437 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f38ab9a-69db-408e-aea4-ee4fa814c12c service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:01.282788 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 852/1704] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:00 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:52:01.284021 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38ab9a-69db-408e-aea4-ee4fa814c12c service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:01.286753 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1f38ab9a-69db-408e-aea4-ee4fa814c12c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:01.360657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 853/1705] 10.222.0.97 () {58 vars in 1009 bytes} [Tue Jul 26 14:52:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:01.362201 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60188203-2be7-4d90-9afe-65f27f41e39c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:01.362499 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60188203-2be7-4d90-9afe-65f27f41e39c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:01.362780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60188203-2be7-4d90-9afe-65f27f41e39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:01.366143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:01.412734 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-60188203-2be7-4d90-9afe-65f27f41e39c None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:01.416537 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-60188203-2be7-4d90-9afe-65f27f41e39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:01.422923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38ab9a-69db-408e-aea4-ee4fa814c12c service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:01.436401 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a00b20-4297-421b-8ded-5b02645f9ee7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:01.437209 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a00b20-4297-421b-8ded-5b02645f9ee7 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:01.437568 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a00b20-4297-421b-8ded-5b02645f9ee7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:01.438069 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a00b20-4297-421b-8ded-5b02645f9ee7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:01.455682 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a00b20-4297-421b-8ded-5b02645f9ee7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:01.459262 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a00b20-4297-421b-8ded-5b02645f9ee7 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:01.493085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 853/1706] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:52:01.546566 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 854/1707] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:01.665343 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a00b20-4297-421b-8ded-5b02645f9ee7 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:01.681202 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a00b20-4297-421b-8ded-5b02645f9ee7 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:01.692777 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797be10b-9176-4726-9438-7635a96b9cce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:01.696511 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797be10b-9176-4726-9438-7635a96b9cce service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:01.699658 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797be10b-9176-4726-9438-7635a96b9cce service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:01.700198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797be10b-9176-4726-9438-7635a96b9cce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:01.719021 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60188203-2be7-4d90-9afe-65f27f41e39c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:01.720597 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-60188203-2be7-4d90-9afe-65f27f41e39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:01.740295 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60188203-2be7-4d90-9afe-65f27f41e39c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:01.750437 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797be10b-9176-4726-9438-7635a96b9cce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:01.752829 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797be10b-9176-4726-9438-7635a96b9cce service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:01.753637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479983c6-0b26-4977-8488-594a5eb3c39e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:01.754528 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479983c6-0b26-4977-8488-594a5eb3c39e service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:01.759023 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479983c6-0b26-4977-8488-594a5eb3c39e service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:01.759326 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479983c6-0b26-4977-8488-594a5eb3c39e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:01.779674 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479983c6-0b26-4977-8488-594a5eb3c39e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:01.792215 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479983c6-0b26-4977-8488-594a5eb3c39e service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:01.798049 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 854/1708] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:52:01.815911 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 855/1709] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:52:02.675024 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51e5ec-7013-4961-99e2-7d36cb1ab8a8 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:02.675614 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51e5ec-7013-4961-99e2-7d36cb1ab8a8 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:02.675900 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51e5ec-7013-4961-99e2-7d36cb1ab8a8 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:02.677427 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 855/1710] 10.222.0.97 () {54 vars in 961 bytes} [Tue Jul 26 14:52:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:52:02.690389 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72652cb0-6384-4d44-80bb-e93932471ce5 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:02.690868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72652cb0-6384-4d44-80bb-e93932471ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:02.690940 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72652cb0-6384-4d44-80bb-e93932471ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:02.737028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-72652cb0-6384-4d44-80bb-e93932471ce5 None None] MFA Rules not processed for user `252c2d2939134416a3e8efb254f03c5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:02.737028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-72652cb0-6384-4d44-80bb-e93932471ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:02.932197 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b51e5ec-7013-4961-99e2-7d36cb1ab8a8 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:02.933245 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5b51e5ec-7013-4961-99e2-7d36cb1ab8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:02.943104 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:02.965576 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 856/1711] 10.222.0.97 () {58 vars in 1058 bytes} [Tue Jul 26 14:52:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:02.974583 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72652cb0-6384-4d44-80bb-e93932471ce5 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:03.008440 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:03.009551 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72652cb0-6384-4d44-80bb-e93932471ce5 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:03.058355 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febac580-03bf-4d6d-b95f-1792c70b8d94 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '252c2d2939134416a3e8efb254f03c5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:03.059242 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febac580-03bf-4d6d-b95f-1792c70b8d94 service placement] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:03.059694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febac580-03bf-4d6d-b95f-1792c70b8d94 service placement] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:03.060110 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febac580-03bf-4d6d-b95f-1792c70b8d94 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:03.064391 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-febac580-03bf-4d6d-b95f-1792c70b8d94 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:03.112173 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febac580-03bf-4d6d-b95f-1792c70b8d94 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:03.121305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febac580-03bf-4d6d-b95f-1792c70b8d94 service placement] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:03.154265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b51e5ec-7013-4961-99e2-7d36cb1ab8a8 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:03.159601 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 857/1712] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:52:03.165449 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdda415-89ea-4ae8-99e2-792220d6cd1f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '252c2d2939134416a3e8efb254f03c5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:03.170038 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdda415-89ea-4ae8-99e2-792220d6cd1f service placement] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:03.177908 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdda415-89ea-4ae8-99e2-792220d6cd1f service placement] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:03.177908 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdda415-89ea-4ae8-99e2-792220d6cd1f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:03.190706 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdda415-89ea-4ae8-99e2-792220d6cd1f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:03.192837 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdda415-89ea-4ae8-99e2-792220d6cd1f service placement] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:03.225447 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 856/1713] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:52:03.401116 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febac580-03bf-4d6d-b95f-1792c70b8d94 service placement] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:03.402749 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-febac580-03bf-4d6d-b95f-1792c70b8d94 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:03.442318 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febac580-03bf-4d6d-b95f-1792c70b8d94 service placement] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:03.467613 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5ca88-90d9-4667-a30d-6690304494f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:03.467613 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5ca88-90d9-4667-a30d-6690304494f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:03.467613 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5ca88-90d9-4667-a30d-6690304494f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:03.467613 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5ca88-90d9-4667-a30d-6690304494f6 admin admin] PATH_INFO: `/v3/projects/1f77b8972e364573ba62facb721fa7bd` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:03.468918 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d5ca88-90d9-4667-a30d-6690304494f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f77b8972e364573ba62facb721fa7bd)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:03.471348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d5ca88-90d9-4667-a30d-6690304494f6 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:03.653532 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59e6ec-aef2-4dea-b358-9d4e609e5e92 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:03.653829 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59e6ec-aef2-4dea-b358-9d4e609e5e92 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:03.654122 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59e6ec-aef2-4dea-b358-9d4e609e5e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:03.680128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f7d5ca88-90d9-4667-a30d-6690304494f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f77b8972e364573ba62facb721fa7bd', 'request_id': 'req-f7d5ca88-90d9-4667-a30d-6690304494f6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:03.695416 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f7d5ca88-90d9-4667-a30d-6690304494f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f77b8972e364573ba62facb721fa7bd', 'request_id': 'req-f7d5ca88-90d9-4667-a30d-6690304494f6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:03.723525 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-5a59e6ec-aef2-4dea-b358-9d4e609e5e92 None None] MFA Rules not processed for user `19c0408b4321495eabd4738bbb8b711b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:03.730251 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5a59e6ec-aef2-4dea-b358-9d4e609e5e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:03.830983 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f7d5ca88-90d9-4667-a30d-6690304494f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f77b8972e364573ba62facb721fa7bd', 'request_id': 'req-f7d5ca88-90d9-4667-a30d-6690304494f6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:03.874714 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f7d5ca88-90d9-4667-a30d-6690304494f6 admin admin] The token cache is being invalidate because project 1f77b8972e364573ba62facb721fa7bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:52:03.874714 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f7d5ca88-90d9-4667-a30d-6690304494f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:03.875340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f7d5ca88-90d9-4667-a30d-6690304494f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:03.880458 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 858/1714] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:52:03 2022] DELETE /identity/v3/projects/1f77b8972e364573ba62facb721fa7bd => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:03.886585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 857/1715] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:03.886585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c4f32-51b7-4174-88de-8cb02be7fa03 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:03.886585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c4f32-51b7-4174-88de-8cb02be7fa03 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:03.886585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c4f32-51b7-4174-88de-8cb02be7fa03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:03.888443 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5ca88-90d9-4667-a30d-6690304494f6 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:03.900207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f7d5ca88-90d9-4667-a30d-6690304494f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:03.983417 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-725c4f32-51b7-4174-88de-8cb02be7fa03 None None] MFA Rules not processed for user `e0615d20057342cfbd16aa0ec94d1979`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:03.987297 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-725c4f32-51b7-4174-88de-8cb02be7fa03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:03.999952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:04.160035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5ca88-90d9-4667-a30d-6690304494f6 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:04.171869 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d175c7d-9120-4517-a389-33c04f2ea8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:04.172685 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d175c7d-9120-4517-a389-33c04f2ea8f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:04.172970 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d175c7d-9120-4517-a389-33c04f2ea8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:04.173395 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d175c7d-9120-4517-a389-33c04f2ea8f3 admin admin] PATH_INFO: `/v3/users/0026e58f974d419b8cecf878ab00a38c` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:04.176575 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d175c7d-9120-4517-a389-33c04f2ea8f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0026e58f974d419b8cecf878ab00a38c)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:04.179131 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d175c7d-9120-4517-a389-33c04f2ea8f3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:04.228226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 858/1716] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:04.559639 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8d175c7d-9120-4517-a389-33c04f2ea8f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0026e58f974d419b8cecf878ab00a38c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:04.561223 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8d175c7d-9120-4517-a389-33c04f2ea8f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0026e58f974d419b8cecf878ab00a38c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:04.590854 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8d175c7d-9120-4517-a389-33c04f2ea8f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0026e58f974d419b8cecf878ab00a38c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:04.612480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8d175c7d-9120-4517-a389-33c04f2ea8f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0026e58f974d419b8cecf878ab00a38c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:04.613264 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8d175c7d-9120-4517-a389-33c04f2ea8f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0026e58f974d419b8cecf878ab00a38c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:04.699262 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 859/1717] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:52:03 2022] DELETE /identity/v3/users/0026e58f974d419b8cecf878ab00a38c => generated 0 bytes in 811 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:05.858127 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:06.031743 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:06.281313 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c4f32-51b7-4174-88de-8cb02be7fa03 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:06.325202 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c4f32-51b7-4174-88de-8cb02be7fa03 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:06.609637 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b1fd6-566c-4e26-8a75-4d30db09c42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:06.623774 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b1fd6-566c-4e26-8a75-4d30db09c42b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:06.631007 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b1fd6-566c-4e26-8a75-4d30db09c42b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:06.644961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b1fd6-566c-4e26-8a75-4d30db09c42b admin admin] PATH_INFO: `/v3/projects/52f16ee6af45442ca660a09f20cf5b89` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:06.649649 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b1fd6-566c-4e26-8a75-4d30db09c42b admin admin] RBAC: Authorizing `identity:delete_project(project_id=52f16ee6af45442ca660a09f20cf5b89)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:06.662939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b1fd6-566c-4e26-8a75-4d30db09c42b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:06.903222 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-338b1fd6-566c-4e26-8a75-4d30db09c42b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52f16ee6af45442ca660a09f20cf5b89', 'request_id': 'req-338b1fd6-566c-4e26-8a75-4d30db09c42b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:06.911611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-338b1fd6-566c-4e26-8a75-4d30db09c42b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52f16ee6af45442ca660a09f20cf5b89', 'request_id': 'req-338b1fd6-566c-4e26-8a75-4d30db09c42b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:07.110156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-338b1fd6-566c-4e26-8a75-4d30db09c42b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52f16ee6af45442ca660a09f20cf5b89', 'request_id': 'req-338b1fd6-566c-4e26-8a75-4d30db09c42b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:07.140724 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-338b1fd6-566c-4e26-8a75-4d30db09c42b admin admin] The token cache is being invalidate because project 52f16ee6af45442ca660a09f20cf5b89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:52:07.141250 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-338b1fd6-566c-4e26-8a75-4d30db09c42b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:07.157660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-338b1fd6-566c-4e26-8a75-4d30db09c42b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:07.161423 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 859/1718] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:52:06 2022] DELETE /identity/v3/projects/52f16ee6af45442ca660a09f20cf5b89 => generated 0 bytes in 881 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:07.167370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283e8c3-5793-4b83-89cd-389f0dbff5d3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:07.168189 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283e8c3-5793-4b83-89cd-389f0dbff5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:07.168189 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283e8c3-5793-4b83-89cd-389f0dbff5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:07.213419 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f283e8c3-5793-4b83-89cd-389f0dbff5d3 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:07.214649 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f283e8c3-5793-4b83-89cd-389f0dbff5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:07.333395 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 860/1719] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:07.342600 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924921f3-0a72-4233-9a25-1750ea1f1d6e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:07.342888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924921f3-0a72-4233-9a25-1750ea1f1d6e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:07.343375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924921f3-0a72-4233-9a25-1750ea1f1d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:07.364808 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-924921f3-0a72-4233-9a25-1750ea1f1d6e None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:07.366379 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-924921f3-0a72-4233-9a25-1750ea1f1d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:07.378538 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 860/1720] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:07.384437 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283e8c3-5793-4b83-89cd-389f0dbff5d3 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:07.397422 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283e8c3-5793-4b83-89cd-389f0dbff5d3 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:07.403269 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:07.420700 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35cab4f-a726-46de-854c-94e6dbe5e3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:07.421852 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35cab4f-a726-46de-854c-94e6dbe5e3b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:07.422376 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35cab4f-a726-46de-854c-94e6dbe5e3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:07.422844 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35cab4f-a726-46de-854c-94e6dbe5e3b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:07.423813 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35cab4f-a726-46de-854c-94e6dbe5e3b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:52:07.424279 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35cab4f-a726-46de-854c-94e6dbe5e3b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:07.425995 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35cab4f-a726-46de-854c-94e6dbe5e3b3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:07.448308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924921f3-0a72-4233-9a25-1750ea1f1d6e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:07.458265 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 861/1721] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:52:07.462520 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924921f3-0a72-4233-9a25-1750ea1f1d6e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:07.473016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab67cc28-e0bc-4f8d-be88-1cbb845f32b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:07.474146 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67cc28-e0bc-4f8d-be88-1cbb845f32b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:07.474146 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67cc28-e0bc-4f8d-be88-1cbb845f32b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:07.475139 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67cc28-e0bc-4f8d-be88-1cbb845f32b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:07.476071 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab67cc28-e0bc-4f8d-be88-1cbb845f32b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:07.481002 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab67cc28-e0bc-4f8d-be88-1cbb845f32b4 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:07.627948 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 861/1722] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:52:07 2022] POST /identity/v3/projects => generated 379 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:52:07.632569 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35cab4f-a726-46de-854c-94e6dbe5e3b3 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:07.646768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35cab4f-a726-46de-854c-94e6dbe5e3b3 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:07.800944 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5426c5ec-ec12-405c-a50d-4e6f607f0dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:07.801957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426c5ec-ec12-405c-a50d-4e6f607f0dad admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:07.802391 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426c5ec-ec12-405c-a50d-4e6f607f0dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:07.802814 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426c5ec-ec12-405c-a50d-4e6f607f0dad admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:07.803656 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5426c5ec-ec12-405c-a50d-4e6f607f0dad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:07.805863 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5426c5ec-ec12-405c-a50d-4e6f607f0dad admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:07.944171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 862/1723] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:52:07 2022] POST /identity/v3/users => generated 290 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:52:07.946806 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab67cc28-e0bc-4f8d-be88-1cbb845f32b4 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:07.973253 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab67cc28-e0bc-4f8d-be88-1cbb845f32b4 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:07.985372 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63bf4cc-df9d-4c4d-986c-26bd4f68f5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:07.986682 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63bf4cc-df9d-4c4d-986c-26bd4f68f5f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:07.988059 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63bf4cc-df9d-4c4d-986c-26bd4f68f5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:07.988468 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63bf4cc-df9d-4c4d-986c-26bd4f68f5f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:07.992252 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63bf4cc-df9d-4c4d-986c-26bd4f68f5f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:07.996767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63bf4cc-df9d-4c4d-986c-26bd4f68f5f6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:08.027554 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 862/1724] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:52:08.028247 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5426c5ec-ec12-405c-a50d-4e6f607f0dad admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:08.040428 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5426c5ec-ec12-405c-a50d-4e6f607f0dad admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:08.046175 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba397d4-046e-47db-8613-1e4a50ca4671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:08.047115 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba397d4-046e-47db-8613-1e4a50ca4671 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:08.047584 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba397d4-046e-47db-8613-1e4a50ca4671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:08.048156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba397d4-046e-47db-8613-1e4a50ca4671 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:08.049191 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba397d4-046e-47db-8613-1e4a50ca4671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:08.051284 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba397d4-046e-47db-8613-1e4a50ca4671 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:08.070093 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63bf4cc-df9d-4c4d-986c-26bd4f68f5f6 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:08.081933 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 863/1725] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:52:08.082890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63bf4cc-df9d-4c4d-986c-26bd4f68f5f6 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:08.138662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8bb911-1467-460d-a5f9-6a59a99818f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:08.139369 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8bb911-1467-460d-a5f9-6a59a99818f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:08.139813 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8bb911-1467-460d-a5f9-6a59a99818f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:08.140103 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8bb911-1467-460d-a5f9-6a59a99818f1 admin admin] PATH_INFO: `/v3/projects/4edd0f660b88417abe04de94a68a7055/users/124ecd0a9f614786976489013f04d3cf/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:08.169945 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8bb911-1467-460d-a5f9-6a59a99818f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4edd0f660b88417abe04de94a68a7055, user_id=124ecd0a9f614786976489013f04d3cf, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:08.173596 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8bb911-1467-460d-a5f9-6a59a99818f1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:08.272458 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 863/1726] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:52:08 2022] PUT /identity/v3/projects/4edd0f660b88417abe04de94a68a7055/users/124ecd0a9f614786976489013f04d3cf/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:08.286842 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295da08-e3c4-4eb9-a276-76693311bd18 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:08.287363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295da08-e3c4-4eb9-a276-76693311bd18 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:08.287646 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295da08-e3c4-4eb9-a276-76693311bd18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:08.341267 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-1295da08-e3c4-4eb9-a276-76693311bd18 None None] MFA Rules not processed for user `124ecd0a9f614786976489013f04d3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:08.342821 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1295da08-e3c4-4eb9-a276-76693311bd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:08.414434 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 864/1727] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:08.419151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8bb911-1467-460d-a5f9-6a59a99818f1 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:08.487040 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8bb911-1467-460d-a5f9-6a59a99818f1 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:08.492438 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4f508-b7f9-445f-a54d-7bd0dd1e85b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:08.493124 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4f508-b7f9-445f-a54d-7bd0dd1e85b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:08.493408 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4f508-b7f9-445f-a54d-7bd0dd1e85b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:08.493684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4f508-b7f9-445f-a54d-7bd0dd1e85b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:08.494355 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed4f508-b7f9-445f-a54d-7bd0dd1e85b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:08.496323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed4f508-b7f9-445f-a54d-7bd0dd1e85b4 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:08.663140 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 864/1728] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:52:08 2022] POST /identity/v3/projects => generated 379 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:52:08.665213 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:08.665507 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1295da08-e3c4-4eb9-a276-76693311bd18 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:08.680624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1295da08-e3c4-4eb9-a276-76693311bd18 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:08.741918 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd1b4e-ef66-4730-bf29-19ec8f72704f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:08.742582 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd1b4e-ef66-4730-bf29-19ec8f72704f admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:08.742879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd1b4e-ef66-4730-bf29-19ec8f72704f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:08.743846 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd1b4e-ef66-4730-bf29-19ec8f72704f admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:08.744008 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd1b4e-ef66-4730-bf29-19ec8f72704f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:08.746757 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd1b4e-ef66-4730-bf29-19ec8f72704f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:08.859504 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 865/1729] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:52:08 2022] POST /identity/v3/users => generated 290 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:52:08.869192 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4f508-b7f9-445f-a54d-7bd0dd1e85b4 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:08.880304 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4f508-b7f9-445f-a54d-7bd0dd1e85b4 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:08.884921 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e75a5-8d84-436a-8d02-01a38ae2fb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:08.885555 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e75a5-8d84-436a-8d02-01a38ae2fb19 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:08.886006 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e75a5-8d84-436a-8d02-01a38ae2fb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:08.886434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e75a5-8d84-436a-8d02-01a38ae2fb19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:08.887301 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e75a5-8d84-436a-8d02-01a38ae2fb19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:08.890053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e75a5-8d84-436a-8d02-01a38ae2fb19 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:08.912946 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 865/1730] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:52:08.917155 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd1b4e-ef66-4730-bf29-19ec8f72704f admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:08.928647 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd1b4e-ef66-4730-bf29-19ec8f72704f admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:08.933586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367476b6-2a90-4d8f-be84-cc3b3be5a682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:08.934356 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367476b6-2a90-4d8f-be84-cc3b3be5a682 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:08.934738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367476b6-2a90-4d8f-be84-cc3b3be5a682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:08.935119 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367476b6-2a90-4d8f-be84-cc3b3be5a682 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:08.936030 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367476b6-2a90-4d8f-be84-cc3b3be5a682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:08.937792 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367476b6-2a90-4d8f-be84-cc3b3be5a682 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:08.950822 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 866/1731] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:52:08.955020 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e75a5-8d84-436a-8d02-01a38ae2fb19 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:08.965579 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e75a5-8d84-436a-8d02-01a38ae2fb19 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:08.970117 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abfcadc-f7b3-4fe0-b23e-c1d5283d069a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:08.970760 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abfcadc-f7b3-4fe0-b23e-c1d5283d069a admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:08.971043 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abfcadc-f7b3-4fe0-b23e-c1d5283d069a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:08.971321 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abfcadc-f7b3-4fe0-b23e-c1d5283d069a admin admin] PATH_INFO: `/v3/projects/f4adf14e9ff44720a77761121429eac4/users/1c6ae4cabd9c4977befb768cc94e9fd3/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:08.985899 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abfcadc-f7b3-4fe0-b23e-c1d5283d069a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4adf14e9ff44720a77761121429eac4, user_id=1c6ae4cabd9c4977befb768cc94e9fd3, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:08.988628 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abfcadc-f7b3-4fe0-b23e-c1d5283d069a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:09.065054 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 866/1732] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:52:08 2022] PUT /identity/v3/projects/f4adf14e9ff44720a77761121429eac4/users/1c6ae4cabd9c4977befb768cc94e9fd3/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:09.076106 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a806ad4-fe91-4209-800a-518d4c01c879 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:09.076395 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a806ad4-fe91-4209-800a-518d4c01c879 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:09.076683 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a806ad4-fe91-4209-800a-518d4c01c879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:09.107675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-9a806ad4-fe91-4209-800a-518d4c01c879 None None] MFA Rules not processed for user `1c6ae4cabd9c4977befb768cc94e9fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:09.108976 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9a806ad4-fe91-4209-800a-518d4c01c879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:09.208782 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 867/1733] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:09.211076 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abfcadc-f7b3-4fe0-b23e-c1d5283d069a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:09.327237 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abfcadc-f7b3-4fe0-b23e-c1d5283d069a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:09.332368 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e11246-ae6a-435b-b607-6c5663ba1d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:09.333344 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e11246-ae6a-435b-b607-6c5663ba1d14 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:09.333631 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e11246-ae6a-435b-b607-6c5663ba1d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:09.333923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e11246-ae6a-435b-b607-6c5663ba1d14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:09.344621 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e11246-ae6a-435b-b607-6c5663ba1d14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:09.352059 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e11246-ae6a-435b-b607-6c5663ba1d14 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:09.545067 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 867/1734] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:52:09 2022] POST /identity/v3/projects => generated 377 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:52:09.548944 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a806ad4-fe91-4209-800a-518d4c01c879 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:09.562915 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a806ad4-fe91-4209-800a-518d4c01c879 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:09.733922 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb3e04-2bf4-4df8-bb4b-6bc33a7ab8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:09.747788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb3e04-2bf4-4df8-bb4b-6bc33a7ab8d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:09.751809 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb3e04-2bf4-4df8-bb4b-6bc33a7ab8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:09.752342 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:09.753479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb3e04-2bf4-4df8-bb4b-6bc33a7ab8d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:09.754366 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb3e04-2bf4-4df8-bb4b-6bc33a7ab8d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:09.760679 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb3e04-2bf4-4df8-bb4b-6bc33a7ab8d3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:09.924866 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 868/1735] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:52:09 2022] POST /identity/v3/users => generated 295 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:52:09.932068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e11246-ae6a-435b-b607-6c5663ba1d14 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:09.953583 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e11246-ae6a-435b-b607-6c5663ba1d14 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:09.979016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e33095-5395-4a01-91e7-382da0667f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:09.979709 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e33095-5395-4a01-91e7-382da0667f4b admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:09.980003 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e33095-5395-4a01-91e7-382da0667f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:09.980284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e33095-5395-4a01-91e7-382da0667f4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:09.981890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e33095-5395-4a01-91e7-382da0667f4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:09.985996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e33095-5395-4a01-91e7-382da0667f4b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:10.008403 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 868/1736] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:09 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:52:10.012798 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb3e04-2bf4-4df8-bb4b-6bc33a7ab8d3 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:10.040789 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb3e04-2bf4-4df8-bb4b-6bc33a7ab8d3 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:10.052794 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106795e9-253a-4d84-b8bd-45cc68a4cb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:10.053736 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106795e9-253a-4d84-b8bd-45cc68a4cb84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:10.054027 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106795e9-253a-4d84-b8bd-45cc68a4cb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:10.054328 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106795e9-253a-4d84-b8bd-45cc68a4cb84 admin admin] PATH_INFO: `/v3/projects/5a13b65a51be4d4eb058f172bdfc8ffe/users/7a4d3f56e07e475c84a6910be016ff6a/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:10.090397 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106795e9-253a-4d84-b8bd-45cc68a4cb84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a13b65a51be4d4eb058f172bdfc8ffe, user_id=7a4d3f56e07e475c84a6910be016ff6a, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:10.093666 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106795e9-253a-4d84-b8bd-45cc68a4cb84 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:10.213123 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 869/1737] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:52:10 2022] PUT /identity/v3/projects/5a13b65a51be4d4eb058f172bdfc8ffe/users/7a4d3f56e07e475c84a6910be016ff6a/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:10.219058 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e33095-5395-4a01-91e7-382da0667f4b admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:10.231219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e33095-5395-4a01-91e7-382da0667f4b admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:10.483039 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c70b36-6884-4935-8079-d6870e6e54c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:10.483773 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c70b36-6884-4935-8079-d6870e6e54c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:10.488776 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c70b36-6884-4935-8079-d6870e6e54c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:10.488776 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c70b36-6884-4935-8079-d6870e6e54c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:10.488776 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c70b36-6884-4935-8079-d6870e6e54c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:10.488776 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c70b36-6884-4935-8079-d6870e6e54c1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:10.528486 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 869/1738] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:52:10.529714 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106795e9-253a-4d84-b8bd-45cc68a4cb84 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:10.568415 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106795e9-253a-4d84-b8bd-45cc68a4cb84 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:10.578352 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fbfbcc-5ec1-4082-b4c6-ad0501185090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:10.578352 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbfbcc-5ec1-4082-b4c6-ad0501185090 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:10.578352 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbfbcc-5ec1-4082-b4c6-ad0501185090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:10.578352 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbfbcc-5ec1-4082-b4c6-ad0501185090 admin admin] PATH_INFO: `/v3/projects/5a13b65a51be4d4eb058f172bdfc8ffe/users/7a4d3f56e07e475c84a6910be016ff6a/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:10.579576 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fbfbcc-5ec1-4082-b4c6-ad0501185090 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a13b65a51be4d4eb058f172bdfc8ffe, user_id=7a4d3f56e07e475c84a6910be016ff6a, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:10.581110 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fbfbcc-5ec1-4082-b4c6-ad0501185090 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:10.637906 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 870/1739] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:52:10 2022] PUT /identity/v3/projects/5a13b65a51be4d4eb058f172bdfc8ffe/users/7a4d3f56e07e475c84a6910be016ff6a/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:10.649845 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4b6d2-c584-4b0f-b82c-d4491afea080 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:10.650329 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4b6d2-c584-4b0f-b82c-d4491afea080 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:10.650784 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4b6d2-c584-4b0f-b82c-d4491afea080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:10.698768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97369b-d3db-49aa-9992-5eeb4f3489b3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:10.698768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97369b-d3db-49aa-9992-5eeb4f3489b3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:10.698768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97369b-d3db-49aa-9992-5eeb4f3489b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:10.756746 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c7e4b6d2-c584-4b0f-b82c-d4491afea080 None None] MFA Rules not processed for user `7a4d3f56e07e475c84a6910be016ff6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:10.757991 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c7e4b6d2-c584-4b0f-b82c-d4491afea080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:10.833464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-8a97369b-d3db-49aa-9992-5eeb4f3489b3 None None] MFA Rules not processed for user `5179c3c4eaaa41f3beb79ba7f1993478`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:10.847440 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8a97369b-d3db-49aa-9992-5eeb4f3489b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:11.024211 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 870/1740] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:10 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:11.072220 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 871/1741] 10.222.0.97 () {58 vars in 1009 bytes} [Tue Jul 26 14:52:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 26 14:52:11.079163 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e4b6d2-c584-4b0f-b82c-d4491afea080 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:11.084195 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c7e4b6d2-c584-4b0f-b82c-d4491afea080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:11.200069 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e4b6d2-c584-4b0f-b82c-d4491afea080 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:11.208602 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7e301-4bf8-4c5b-90b4-8dd9667326f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:11.210255 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7e301-4bf8-4c5b-90b4-8dd9667326f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:11.210727 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7e301-4bf8-4c5b-90b4-8dd9667326f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:11.211155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7e301-4bf8-4c5b-90b4-8dd9667326f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:11.237682 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f7e301-4bf8-4c5b-90b4-8dd9667326f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:11.240055 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f7e301-4bf8-4c5b-90b4-8dd9667326f5 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:11.272611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 871/1742] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:11 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:52:11.657042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a97369b-d3db-49aa-9992-5eeb4f3489b3 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:11.658453 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8a97369b-d3db-49aa-9992-5eeb4f3489b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:11.704448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7e301-4bf8-4c5b-90b4-8dd9667326f5 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:11.714629 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7e301-4bf8-4c5b-90b4-8dd9667326f5 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:11.720585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c0f962-2791-47c8-ab60-32b849db0c5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:11.722131 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0f962-2791-47c8-ab60-32b849db0c5f service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:11.722526 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0f962-2791-47c8-ab60-32b849db0c5f service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:11.723011 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0f962-2791-47c8-ab60-32b849db0c5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:11.801458 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c0f962-2791-47c8-ab60-32b849db0c5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:11.803109 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c0f962-2791-47c8-ab60-32b849db0c5f service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:11.808004 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a97369b-d3db-49aa-9992-5eeb4f3489b3 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:11.812374 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 872/1743] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:52:11.823781 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b26c940-0428-4082-b418-2cc8208bef49 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85122c4052fd4ba19a2dae185a707caf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:11.824757 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26c940-0428-4082-b418-2cc8208bef49 service swift] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:11.825493 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26c940-0428-4082-b418-2cc8208bef49 service swift] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:11.825917 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26c940-0428-4082-b418-2cc8208bef49 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:11.840956 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b26c940-0428-4082-b418-2cc8208bef49 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:11.843560 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b26c940-0428-4082-b418-2cc8208bef49 service swift] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:11.863918 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 872/1744] 10.222.0.97 () {60 vars in 1384 bytes} [Tue Jul 26 14:52:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 14:52:12.303198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973a2be-3f1d-44bb-9ad9-fb1816eeb929 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:12.304577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973a2be-3f1d-44bb-9ad9-fb1816eeb929 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:12.305035 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973a2be-3f1d-44bb-9ad9-fb1816eeb929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:12.379014 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:12.392672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-d973a2be-3f1d-44bb-9ad9-fb1816eeb929 None None] MFA Rules not processed for user `c81180fb4ff04c1186a21e43fb597a91`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:12.405136 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d973a2be-3f1d-44bb-9ad9-fb1816eeb929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:12.648089 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 873/1745] 10.222.0.99 () {58 vars in 1015 bytes} [Tue Jul 26 14:52:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:12.833052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b26c940-0428-4082-b418-2cc8208bef49 service swift] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:12.835289 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0b26c940-0428-4082-b418-2cc8208bef49 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:12.898145 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b26c940-0428-4082-b418-2cc8208bef49 service swift] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:12.948114 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe3003f-d47c-49d1-b88b-f23814151725 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:12.948922 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3003f-d47c-49d1-b88b-f23814151725 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:12.949217 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3003f-d47c-49d1-b88b-f23814151725 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:12.949501 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3003f-d47c-49d1-b88b-f23814151725 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:12.980796 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973a2be-3f1d-44bb-9ad9-fb1816eeb929 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:12.982471 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d973a2be-3f1d-44bb-9ad9-fb1816eeb929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:12.986037 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe3003f-d47c-49d1-b88b-f23814151725 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:12.989134 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe3003f-d47c-49d1-b88b-f23814151725 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:13.001136 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973a2be-3f1d-44bb-9ad9-fb1816eeb929 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:13.013071 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8d7d5-a7e9-47e3-b3ee-8122c332e346 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:13.013908 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8d7d5-a7e9-47e3-b3ee-8122c332e346 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:13.014315 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8d7d5-a7e9-47e3-b3ee-8122c332e346 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:13.014550 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 873/1746] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:52:13.014791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8d7d5-a7e9-47e3-b3ee-8122c332e346 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:13.039816 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8d7d5-a7e9-47e3-b3ee-8122c332e346 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:13.049242 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8d7d5-a7e9-47e3-b3ee-8122c332e346 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:13.075382 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 874/1747] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:52:13.678910 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:15.119031 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:15.362543 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6229d03-8fa1-4be3-82dd-eb4351355d35 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:15.363022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6229d03-8fa1-4be3-82dd-eb4351355d35 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:15.363117 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6229d03-8fa1-4be3-82dd-eb4351355d35 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:15.364510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 874/1748] 10.222.0.97 () {54 vars in 921 bytes} [Tue Jul 26 14:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:52:15.397390 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe0897-012d-49e0-a8e2-1b50e9d96c57 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:15.397728 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe0897-012d-49e0-a8e2-1b50e9d96c57 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:15.398055 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe0897-012d-49e0-a8e2-1b50e9d96c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:15.502201 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-11fe0897-012d-49e0-a8e2-1b50e9d96c57 None None] MFA Rules not processed for user `266abef6cda5495b897170f92043aa03`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:15.506002 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-11fe0897-012d-49e0-a8e2-1b50e9d96c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:15.724198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 875/1749] 10.222.0.97 () {58 vars in 1018 bytes} [Tue Jul 26 14:52:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:16.205072 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6229d03-8fa1-4be3-82dd-eb4351355d35 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:16.206905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f6229d03-8fa1-4be3-82dd-eb4351355d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:16.344049 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6229d03-8fa1-4be3-82dd-eb4351355d35 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:16.365465 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9795b3-c9d2-4284-90fe-94aefdd05621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:16.366202 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe0897-012d-49e0-a8e2-1b50e9d96c57 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:16.367634 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-11fe0897-012d-49e0-a8e2-1b50e9d96c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:16.368052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9795b3-c9d2-4284-90fe-94aefdd05621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:16.368345 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9795b3-c9d2-4284-90fe-94aefdd05621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:16.368633 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9795b3-c9d2-4284-90fe-94aefdd05621 admin admin] PATH_INFO: `/v3/users/239fed388926425fa8987ad9fac79394` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:16.369959 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9795b3-c9d2-4284-90fe-94aefdd05621 admin admin] RBAC: Authorizing `identity:delete_user(user_id=239fed388926425fa8987ad9fac79394)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:16.372766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9795b3-c9d2-4284-90fe-94aefdd05621 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:16.393520 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe0897-012d-49e0-a8e2-1b50e9d96c57 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:16.406168 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f07a38-5d4a-4377-b782-f1dc84b321bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:16.409074 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f07a38-5d4a-4377-b782-f1dc84b321bb service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:16.409074 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f07a38-5d4a-4377-b782-f1dc84b321bb service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:16.409074 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f07a38-5d4a-4377-b782-f1dc84b321bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:16.428777 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f07a38-5d4a-4377-b782-f1dc84b321bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:16.430547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f07a38-5d4a-4377-b782-f1dc84b321bb service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:16.466398 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 876/1750] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:52:16.904586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1e9795b3-c9d2-4284-90fe-94aefdd05621 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '239fed388926425fa8987ad9fac79394'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:16.906253 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1e9795b3-c9d2-4284-90fe-94aefdd05621 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '239fed388926425fa8987ad9fac79394'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:16.949756 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1e9795b3-c9d2-4284-90fe-94aefdd05621 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '239fed388926425fa8987ad9fac79394'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:16.959618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1e9795b3-c9d2-4284-90fe-94aefdd05621 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '239fed388926425fa8987ad9fac79394'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:16.965947 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1e9795b3-c9d2-4284-90fe-94aefdd05621 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '239fed388926425fa8987ad9fac79394'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:17.149396 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 875/1751] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:52:16 2022] DELETE /identity/v3/users/239fed388926425fa8987ad9fac79394 => generated 0 bytes in 945 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:17.154168 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a82b0-8274-42df-b17b-855fcdf35896 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:17.154462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a82b0-8274-42df-b17b-855fcdf35896 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:17.154950 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a82b0-8274-42df-b17b-855fcdf35896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:17.213822 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:17.228244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ed1a82b0-8274-42df-b17b-855fcdf35896 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:17.233476 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ed1a82b0-8274-42df-b17b-855fcdf35896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:17.345403 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:17.368760 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 877/1752] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:17.407562 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9795b3-c9d2-4284-90fe-94aefdd05621 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:17.410539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1e9795b3-c9d2-4284-90fe-94aefdd05621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:17.484639 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9795b3-c9d2-4284-90fe-94aefdd05621 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:17.489464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427120d8-8af6-4967-b522-d470c797230d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:17.490908 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427120d8-8af6-4967-b522-d470c797230d service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:17.491355 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427120d8-8af6-4967-b522-d470c797230d service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:17.491808 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427120d8-8af6-4967-b522-d470c797230d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:17.505268 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427120d8-8af6-4967-b522-d470c797230d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:17.507317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427120d8-8af6-4967-b522-d470c797230d service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:17.525054 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 876/1753] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:52:17.673243 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2584d9-f478-471b-9ca5-bf336b43875e None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:17.673796 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2584d9-f478-471b-9ca5-bf336b43875e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:17.674236 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2584d9-f478-471b-9ca5-bf336b43875e None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:17.676170 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 878/1754] 10.222.0.97 () {54 vars in 921 bytes} [Tue Jul 26 14:52:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:52:17.695592 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd231dc-ddd8-42e6-be76-a5e9014100e3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:17.696747 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd231dc-ddd8-42e6-be76-a5e9014100e3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:17.697093 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd231dc-ddd8-42e6-be76-a5e9014100e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:17.736881 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-5fd231dc-ddd8-42e6-be76-a5e9014100e3 None None] MFA Rules not processed for user `266abef6cda5495b897170f92043aa03`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:17.740037 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5fd231dc-ddd8-42e6-be76-a5e9014100e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:17.908766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 877/1755] 10.222.0.97 () {58 vars in 1018 bytes} [Tue Jul 26 14:52:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:52:18.953396 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2584d9-f478-471b-9ca5-bf336b43875e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:18.955193 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-cb2584d9-f478-471b-9ca5-bf336b43875e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:18.977409 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2584d9-f478-471b-9ca5-bf336b43875e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:19.003228 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd2429-632c-4ea0-9f5f-69b52f47b374 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:19.015563 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd2429-632c-4ea0-9f5f-69b52f47b374 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:19.016311 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd2429-632c-4ea0-9f5f-69b52f47b374 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:19.019279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd2429-632c-4ea0-9f5f-69b52f47b374 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:19.074427 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd2429-632c-4ea0-9f5f-69b52f47b374 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:19.077874 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd2429-632c-4ea0-9f5f-69b52f47b374 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:19.091467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 879/1756] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:52:19.931009 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd231dc-ddd8-42e6-be76-a5e9014100e3 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:19.933524 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5fd231dc-ddd8-42e6-be76-a5e9014100e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:19.986010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd231dc-ddd8-42e6-be76-a5e9014100e3 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:19.997680 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c120b9a-3270-4fdd-b7e6-73376489cdbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:19.998353 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c120b9a-3270-4fdd-b7e6-73376489cdbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:19.998640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c120b9a-3270-4fdd-b7e6-73376489cdbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:19.999175 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c120b9a-3270-4fdd-b7e6-73376489cdbf admin admin] PATH_INFO: `/v3/projects/09d5ca5e2ba947e7acc7ceb298e2aa85` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:20.000449 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c120b9a-3270-4fdd-b7e6-73376489cdbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=09d5ca5e2ba947e7acc7ceb298e2aa85)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:20.002528 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c120b9a-3270-4fdd-b7e6-73376489cdbf admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:20.077613 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd2429-632c-4ea0-9f5f-69b52f47b374 service nova] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:20.081173 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-08dd2429-632c-4ea0-9f5f-69b52f47b374 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:20.112812 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd2429-632c-4ea0-9f5f-69b52f47b374 service nova] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:20.125783 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98163baa-5a31-4814-8705-11fe30ad26bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:20.127649 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98163baa-5a31-4814-8705-11fe30ad26bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:20.127921 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98163baa-5a31-4814-8705-11fe30ad26bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:20.130362 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98163baa-5a31-4814-8705-11fe30ad26bc admin admin] PATH_INFO: `/v3/users/124ecd0a9f614786976489013f04d3cf` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:20.132958 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98163baa-5a31-4814-8705-11fe30ad26bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=124ecd0a9f614786976489013f04d3cf)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:20.164009 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98163baa-5a31-4814-8705-11fe30ad26bc admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:20.180346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:20.196514 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7c120b9a-3270-4fdd-b7e6-73376489cdbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09d5ca5e2ba947e7acc7ceb298e2aa85', 'request_id': 'req-7c120b9a-3270-4fdd-b7e6-73376489cdbf'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:20.225206 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7c120b9a-3270-4fdd-b7e6-73376489cdbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09d5ca5e2ba947e7acc7ceb298e2aa85', 'request_id': 'req-7c120b9a-3270-4fdd-b7e6-73376489cdbf'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:20.341645 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7c120b9a-3270-4fdd-b7e6-73376489cdbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09d5ca5e2ba947e7acc7ceb298e2aa85', 'request_id': 'req-7c120b9a-3270-4fdd-b7e6-73376489cdbf'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:20.379379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7c120b9a-3270-4fdd-b7e6-73376489cdbf admin admin] The token cache is being invalidate because project 09d5ca5e2ba947e7acc7ceb298e2aa85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:52:20.382454 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7c120b9a-3270-4fdd-b7e6-73376489cdbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:20.387531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7c120b9a-3270-4fdd-b7e6-73376489cdbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:20.389731 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 878/1757] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:52:19 2022] DELETE /identity/v3/projects/09d5ca5e2ba947e7acc7ceb298e2aa85 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:20.397378 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c120b9a-3270-4fdd-b7e6-73376489cdbf admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:20.400430 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7c120b9a-3270-4fdd-b7e6-73376489cdbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:20.481881 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-98163baa-5a31-4814-8705-11fe30ad26bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '124ecd0a9f614786976489013f04d3cf'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:20.482870 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-98163baa-5a31-4814-8705-11fe30ad26bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '124ecd0a9f614786976489013f04d3cf'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:20.513715 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-98163baa-5a31-4814-8705-11fe30ad26bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '124ecd0a9f614786976489013f04d3cf'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:20.528887 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-98163baa-5a31-4814-8705-11fe30ad26bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '124ecd0a9f614786976489013f04d3cf'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:20.529438 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c120b9a-3270-4fdd-b7e6-73376489cdbf admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:20.530196 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-98163baa-5a31-4814-8705-11fe30ad26bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '124ecd0a9f614786976489013f04d3cf'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:20.544434 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6834df6-6636-441a-b4a6-a3023b903f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:20.545246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6834df6-6636-441a-b4a6-a3023b903f53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:20.545650 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6834df6-6636-441a-b4a6-a3023b903f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:20.546047 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6834df6-6636-441a-b4a6-a3023b903f53 admin admin] PATH_INFO: `/v3/users/b387682a85884879a8cd21b3c7e70cf8` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:20.547632 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6834df6-6636-441a-b4a6-a3023b903f53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b387682a85884879a8cd21b3c7e70cf8)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:20.556228 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6834df6-6636-441a-b4a6-a3023b903f53 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:20.684607 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 880/1758] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:52:20 2022] DELETE /identity/v3/users/124ecd0a9f614786976489013f04d3cf => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:20.689367 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef330e5d-9cb0-45ed-970f-820fbbef03b0 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:20.689658 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef330e5d-9cb0-45ed-970f-820fbbef03b0 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:20.690096 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef330e5d-9cb0-45ed-970f-820fbbef03b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:20.709921 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ef330e5d-9cb0-45ed-970f-820fbbef03b0 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:20.711138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ef330e5d-9cb0-45ed-970f-820fbbef03b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:20.823962 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 881/1759] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:52:20.830445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef330e5d-9cb0-45ed-970f-820fbbef03b0 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:20.832143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ef330e5d-9cb0-45ed-970f-820fbbef03b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:20.894369 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef330e5d-9cb0-45ed-970f-820fbbef03b0 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:20.898365 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f9379-266f-4b06-9e1c-b7679e8d8fa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:20.899793 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f9379-266f-4b06-9e1c-b7679e8d8fa7 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:20.900077 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f9379-266f-4b06-9e1c-b7679e8d8fa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:20.900356 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f9379-266f-4b06-9e1c-b7679e8d8fa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:20.910217 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f9379-266f-4b06-9e1c-b7679e8d8fa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:20.911692 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f9379-266f-4b06-9e1c-b7679e8d8fa7 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:20.916601 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:20.924022 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 882/1760] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:52:20.947062 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a6834df6-6636-441a-b4a6-a3023b903f53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b387682a85884879a8cd21b3c7e70cf8'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:20.948611 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a6834df6-6636-441a-b4a6-a3023b903f53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b387682a85884879a8cd21b3c7e70cf8'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:20.969069 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a6834df6-6636-441a-b4a6-a3023b903f53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b387682a85884879a8cd21b3c7e70cf8'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:20.976690 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a6834df6-6636-441a-b4a6-a3023b903f53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b387682a85884879a8cd21b3c7e70cf8'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:20.978225 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a6834df6-6636-441a-b4a6-a3023b903f53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b387682a85884879a8cd21b3c7e70cf8'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:21.065275 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 879/1761] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:52:20 2022] DELETE /identity/v3/users/b387682a85884879a8cd21b3c7e70cf8 => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:52:21.684981 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f9379-266f-4b06-9e1c-b7679e8d8fa7 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:21.686557 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9e4f9379-266f-4b06-9e1c-b7679e8d8fa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:21.702320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f9379-266f-4b06-9e1c-b7679e8d8fa7 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:21.797151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead51a86-9151-47cf-8184-8f14d9c0ddb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:21.798021 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead51a86-9151-47cf-8184-8f14d9c0ddb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:21.798460 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead51a86-9151-47cf-8184-8f14d9c0ddb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:21.798791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead51a86-9151-47cf-8184-8f14d9c0ddb7 admin admin] PATH_INFO: `/v3/projects/4edd0f660b88417abe04de94a68a7055` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:21.802672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead51a86-9151-47cf-8184-8f14d9c0ddb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4edd0f660b88417abe04de94a68a7055)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:21.805377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead51a86-9151-47cf-8184-8f14d9c0ddb7 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:21.942485 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ead51a86-9151-47cf-8184-8f14d9c0ddb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4edd0f660b88417abe04de94a68a7055', 'request_id': 'req-ead51a86-9151-47cf-8184-8f14d9c0ddb7'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:21.949409 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ead51a86-9151-47cf-8184-8f14d9c0ddb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4edd0f660b88417abe04de94a68a7055', 'request_id': 'req-ead51a86-9151-47cf-8184-8f14d9c0ddb7'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:21.966842 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6834df6-6636-441a-b4a6-a3023b903f53 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:21.968501 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a6834df6-6636-441a-b4a6-a3023b903f53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:21.983505 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6834df6-6636-441a-b4a6-a3023b903f53 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:22.053861 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ead51a86-9151-47cf-8184-8f14d9c0ddb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4edd0f660b88417abe04de94a68a7055', 'request_id': 'req-ead51a86-9151-47cf-8184-8f14d9c0ddb7'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:22.066847 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ead51a86-9151-47cf-8184-8f14d9c0ddb7 admin admin] The token cache is being invalidate because project 4edd0f660b88417abe04de94a68a7055 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:52:22.067371 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ead51a86-9151-47cf-8184-8f14d9c0ddb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:22.069080 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ead51a86-9151-47cf-8184-8f14d9c0ddb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:22.072031 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 883/1762] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:52:21 2022] DELETE /identity/v3/projects/4edd0f660b88417abe04de94a68a7055 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:22.076419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead51a86-9151-47cf-8184-8f14d9c0ddb7 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:22.078003 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ead51a86-9151-47cf-8184-8f14d9c0ddb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:22.137488 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59362d71-b743-4c5e-b639-a16dd836d62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:22.138424 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59362d71-b743-4c5e-b639-a16dd836d62c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:22.138870 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59362d71-b743-4c5e-b639-a16dd836d62c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:22.139273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59362d71-b743-4c5e-b639-a16dd836d62c admin admin] PATH_INFO: `/v3/projects/005a74bb12454f9b99ac6074ed3f18a9` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:22.141429 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59362d71-b743-4c5e-b639-a16dd836d62c admin admin] RBAC: Authorizing `identity:delete_project(project_id=005a74bb12454f9b99ac6074ed3f18a9)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:22.143683 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59362d71-b743-4c5e-b639-a16dd836d62c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:22.215951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead51a86-9151-47cf-8184-8f14d9c0ddb7 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:22.240158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f071f-e80c-431e-b700-e192c4aa8d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:22.242219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f071f-e80c-431e-b700-e192c4aa8d74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:22.242729 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f071f-e80c-431e-b700-e192c4aa8d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:22.243370 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f071f-e80c-431e-b700-e192c4aa8d74 admin admin] PATH_INFO: `/v3/users/1c6ae4cabd9c4977befb768cc94e9fd3` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:22.245053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0f071f-e80c-431e-b700-e192c4aa8d74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c6ae4cabd9c4977befb768cc94e9fd3)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:22.248200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0f071f-e80c-431e-b700-e192c4aa8d74 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:22.295611 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-59362d71-b743-4c5e-b639-a16dd836d62c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '005a74bb12454f9b99ac6074ed3f18a9', 'request_id': 'req-59362d71-b743-4c5e-b639-a16dd836d62c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:22.313094 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-59362d71-b743-4c5e-b639-a16dd836d62c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '005a74bb12454f9b99ac6074ed3f18a9', 'request_id': 'req-59362d71-b743-4c5e-b639-a16dd836d62c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:22.411532 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-59362d71-b743-4c5e-b639-a16dd836d62c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '005a74bb12454f9b99ac6074ed3f18a9', 'request_id': 'req-59362d71-b743-4c5e-b639-a16dd836d62c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:22.435373 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-59362d71-b743-4c5e-b639-a16dd836d62c admin admin] The token cache is being invalidate because project 005a74bb12454f9b99ac6074ed3f18a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:52:22.435791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-59362d71-b743-4c5e-b639-a16dd836d62c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:22.436599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-59362d71-b743-4c5e-b639-a16dd836d62c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:22.438936 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 880/1763] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:52:21 2022] DELETE /identity/v3/projects/005a74bb12454f9b99ac6074ed3f18a9 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:52:22.443757 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71896dfd-5b54-430b-8b51-5b850b23b7b0 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:22.444320 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71896dfd-5b54-430b-8b51-5b850b23b7b0 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:22.444851 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71896dfd-5b54-430b-8b51-5b850b23b7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:22.467824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-71896dfd-5b54-430b-8b51-5b850b23b7b0 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:22.469464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-71896dfd-5b54-430b-8b51-5b850b23b7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:22.578041 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fa0f071f-e80c-431e-b700-e192c4aa8d74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c6ae4cabd9c4977befb768cc94e9fd3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:22.579101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fa0f071f-e80c-431e-b700-e192c4aa8d74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c6ae4cabd9c4977befb768cc94e9fd3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:22.579440 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 881/1764] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:22.587797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a2f8f-ac92-404b-8451-b6bb2eb1902b None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:22.588089 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a2f8f-ac92-404b-8451-b6bb2eb1902b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:22.588375 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a2f8f-ac92-404b-8451-b6bb2eb1902b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:22.593504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fa0f071f-e80c-431e-b700-e192c4aa8d74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c6ae4cabd9c4977befb768cc94e9fd3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:22.615952 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fa0f071f-e80c-431e-b700-e192c4aa8d74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c6ae4cabd9c4977befb768cc94e9fd3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:22.616723 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fa0f071f-e80c-431e-b700-e192c4aa8d74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c6ae4cabd9c4977befb768cc94e9fd3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:22.633966 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-5f6a2f8f-ac92-404b-8451-b6bb2eb1902b None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:22.635203 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5f6a2f8f-ac92-404b-8451-b6bb2eb1902b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:22.649972 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 882/1765] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:22.654344 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a2f8f-ac92-404b-8451-b6bb2eb1902b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:22.675824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a2f8f-ac92-404b-8451-b6bb2eb1902b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:22.683890 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69574645-b82d-4320-96f5-d988d079199b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:22.683890 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69574645-b82d-4320-96f5-d988d079199b admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:22.683890 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69574645-b82d-4320-96f5-d988d079199b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:22.683890 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69574645-b82d-4320-96f5-d988d079199b admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:22.684226 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69574645-b82d-4320-96f5-d988d079199b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:52:22.684477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69574645-b82d-4320-96f5-d988d079199b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:22.686366 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69574645-b82d-4320-96f5-d988d079199b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:22.706544 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 883/1766] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:52:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:52:22.708590 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69574645-b82d-4320-96f5-d988d079199b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:22.721415 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69574645-b82d-4320-96f5-d988d079199b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:22.726150 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b35e04e-34d3-4f3a-9cb4-3d79b5d6bf36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:22.726782 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35e04e-34d3-4f3a-9cb4-3d79b5d6bf36 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:22.727054 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35e04e-34d3-4f3a-9cb4-3d79b5d6bf36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:22.727348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35e04e-34d3-4f3a-9cb4-3d79b5d6bf36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:22.728247 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b35e04e-34d3-4f3a-9cb4-3d79b5d6bf36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:22.730167 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b35e04e-34d3-4f3a-9cb4-3d79b5d6bf36 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:22.878226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 884/1767] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:52:22 2022] DELETE /identity/v3/users/1c6ae4cabd9c4977befb768cc94e9fd3 => generated 0 bytes in 802 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:22.885533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 884/1768] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:52:22 2022] POST /identity/v3/projects => generated 363 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:52:22.896831 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f071f-e80c-431e-b700-e192c4aa8d74 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:22.974161 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f071f-e80c-431e-b700-e192c4aa8d74 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:23.066606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc8570-1680-4aec-860d-4561aa9587b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:23.067289 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc8570-1680-4aec-860d-4561aa9587b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:23.067607 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc8570-1680-4aec-860d-4561aa9587b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:23.067881 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc8570-1680-4aec-860d-4561aa9587b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:23.069102 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc8570-1680-4aec-860d-4561aa9587b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:23.071225 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc8570-1680-4aec-860d-4561aa9587b4 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:23.248039 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 885/1769] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:52:22 2022] POST /identity/v3/users => generated 282 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:52:23.248904 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b35e04e-34d3-4f3a-9cb4-3d79b5d6bf36 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:23.269589 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b35e04e-34d3-4f3a-9cb4-3d79b5d6bf36 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:23.284488 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60021998-2ef1-48a1-bbc5-e0ba7f21f9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:23.285144 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60021998-2ef1-48a1-bbc5-e0ba7f21f9d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:23.285431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60021998-2ef1-48a1-bbc5-e0ba7f21f9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:23.285710 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60021998-2ef1-48a1-bbc5-e0ba7f21f9d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:23.286452 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60021998-2ef1-48a1-bbc5-e0ba7f21f9d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:23.288363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60021998-2ef1-48a1-bbc5-e0ba7f21f9d3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:23.315012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 885/1770] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:52:23.319395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc8570-1680-4aec-860d-4561aa9587b4 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:23.331688 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc8570-1680-4aec-860d-4561aa9587b4 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:23.338473 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9f4789-81d9-4fee-8b36-e4c3b942b1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:23.339256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f4789-81d9-4fee-8b36-e4c3b942b1cc admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:23.339657 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f4789-81d9-4fee-8b36-e4c3b942b1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:23.339981 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f4789-81d9-4fee-8b36-e4c3b942b1cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:23.340792 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9f4789-81d9-4fee-8b36-e4c3b942b1cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:23.343309 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9f4789-81d9-4fee-8b36-e4c3b942b1cc admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:23.363276 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 886/1771] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:52:23.366586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60021998-2ef1-48a1-bbc5-e0ba7f21f9d3 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:23.381019 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60021998-2ef1-48a1-bbc5-e0ba7f21f9d3 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:23.391185 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17db0f2-5854-4d9f-a804-e4c5da72035e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:23.391921 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17db0f2-5854-4d9f-a804-e4c5da72035e admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:23.392208 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17db0f2-5854-4d9f-a804-e4c5da72035e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:23.392486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17db0f2-5854-4d9f-a804-e4c5da72035e admin admin] PATH_INFO: `/v3/projects/2e3803254b504f59bd5c20ffbceeeede/users/122b9d2db6e549ffab33fbe7629754ce/roles/e033961a9363456a8510c34414dd839e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:23.446362 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17db0f2-5854-4d9f-a804-e4c5da72035e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e3803254b504f59bd5c20ffbceeeede, user_id=122b9d2db6e549ffab33fbe7629754ce, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:23.462916 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17db0f2-5854-4d9f-a804-e4c5da72035e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:23.661956 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 886/1772] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:52:23 2022] PUT /identity/v3/projects/2e3803254b504f59bd5c20ffbceeeede/users/122b9d2db6e549ffab33fbe7629754ce/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:23.672087 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81827de6-6c6c-436d-b629-a5ba4423057a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:23.672474 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81827de6-6c6c-436d-b629-a5ba4423057a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:23.672810 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81827de6-6c6c-436d-b629-a5ba4423057a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:23.716543 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-81827de6-6c6c-436d-b629-a5ba4423057a None None] MFA Rules not processed for user `122b9d2db6e549ffab33fbe7629754ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:23.717958 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-81827de6-6c6c-436d-b629-a5ba4423057a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:23.791385 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17db0f2-5854-4d9f-a804-e4c5da72035e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:23.791385 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a17db0f2-5854-4d9f-a804-e4c5da72035e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:23.814480 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 887/1773] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:23 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:23.819985 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81827de6-6c6c-436d-b629-a5ba4423057a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:23.822317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:23.900979 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17db0f2-5854-4d9f-a804-e4c5da72035e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:23.907258 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b1737-71d1-4608-b815-4bd5d7685596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:23.908805 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b1737-71d1-4608-b815-4bd5d7685596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:23.909371 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b1737-71d1-4608-b815-4bd5d7685596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:23.909637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b1737-71d1-4608-b815-4bd5d7685596 admin admin] PATH_INFO: `/v3/projects/f4adf14e9ff44720a77761121429eac4` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:23.910991 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633b1737-71d1-4608-b815-4bd5d7685596 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4adf14e9ff44720a77761121429eac4)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:23.913083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633b1737-71d1-4608-b815-4bd5d7685596 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:23.926662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81827de6-6c6c-436d-b629-a5ba4423057a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:23.932191 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17fb88-4c8d-4fec-b7a3-9521c1010889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:23.932807 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17fb88-4c8d-4fec-b7a3-9521c1010889 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:23.933090 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17fb88-4c8d-4fec-b7a3-9521c1010889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:23.933371 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17fb88-4c8d-4fec-b7a3-9521c1010889 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:23.934037 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac17fb88-4c8d-4fec-b7a3-9521c1010889 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:23.936025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac17fb88-4c8d-4fec-b7a3-9521c1010889 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:24.141831 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 888/1774] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:52:23 2022] POST /identity/v3/projects => generated 363 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:52:24.145891 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17fb88-4c8d-4fec-b7a3-9521c1010889 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:24.153361 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-633b1737-71d1-4608-b815-4bd5d7685596 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4adf14e9ff44720a77761121429eac4', 'request_id': 'req-633b1737-71d1-4608-b815-4bd5d7685596'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:24.158971 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-633b1737-71d1-4608-b815-4bd5d7685596 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4adf14e9ff44720a77761121429eac4', 'request_id': 'req-633b1737-71d1-4608-b815-4bd5d7685596'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:24.208252 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17fb88-4c8d-4fec-b7a3-9521c1010889 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:24.214115 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bc455-b907-4476-a372-94ec6cccf14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:24.214847 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bc455-b907-4476-a372-94ec6cccf14b admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:24.214990 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bc455-b907-4476-a372-94ec6cccf14b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:24.215264 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bc455-b907-4476-a372-94ec6cccf14b admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:24.215998 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6bc455-b907-4476-a372-94ec6cccf14b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:24.218151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6bc455-b907-4476-a372-94ec6cccf14b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:24.260430 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-633b1737-71d1-4608-b815-4bd5d7685596 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4adf14e9ff44720a77761121429eac4', 'request_id': 'req-633b1737-71d1-4608-b815-4bd5d7685596'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:24.274392 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-633b1737-71d1-4608-b815-4bd5d7685596 admin admin] The token cache is being invalidate because project f4adf14e9ff44720a77761121429eac4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:52:24.274687 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-633b1737-71d1-4608-b815-4bd5d7685596 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:24.275370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-633b1737-71d1-4608-b815-4bd5d7685596 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:24.277062 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 887/1775] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:52:23 2022] DELETE /identity/v3/projects/f4adf14e9ff44720a77761121429eac4 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:24.280814 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b1737-71d1-4608-b815-4bd5d7685596 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:24.282324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-633b1737-71d1-4608-b815-4bd5d7685596 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:24.345970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 889/1776] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:52:24 2022] POST /identity/v3/users => generated 288 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:52:24.350289 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bc455-b907-4476-a372-94ec6cccf14b admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:24.352586 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7f6bc455-b907-4476-a372-94ec6cccf14b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:24.356913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b1737-71d1-4608-b815-4bd5d7685596 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:24.359879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84af62d-381e-4e7b-b11c-ba2c9f6aab15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:24.360725 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84af62d-381e-4e7b-b11c-ba2c9f6aab15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:24.361167 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84af62d-381e-4e7b-b11c-ba2c9f6aab15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:24.361597 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84af62d-381e-4e7b-b11c-ba2c9f6aab15 admin admin] PATH_INFO: `/v3/users/7a4d3f56e07e475c84a6910be016ff6a` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:24.363092 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84af62d-381e-4e7b-b11c-ba2c9f6aab15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a4d3f56e07e475c84a6910be016ff6a)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:24.365459 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84af62d-381e-4e7b-b11c-ba2c9f6aab15 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:24.366111 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bc455-b907-4476-a372-94ec6cccf14b admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:24.371977 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e9ba6-3a69-4dc4-88ab-957b07a7a533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:24.372755 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e9ba6-3a69-4dc4-88ab-957b07a7a533 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:24.373050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e9ba6-3a69-4dc4-88ab-957b07a7a533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:24.373333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e9ba6-3a69-4dc4-88ab-957b07a7a533 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:24.374045 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e9ba6-3a69-4dc4-88ab-957b07a7a533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:24.375959 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e9ba6-3a69-4dc4-88ab-957b07a7a533 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:24.387622 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 890/1777] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:52:24.392442 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e9ba6-3a69-4dc4-88ab-957b07a7a533 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:24.403244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e9ba6-3a69-4dc4-88ab-957b07a7a533 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:24.407928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efbefa4-7c9a-4f18-b4e9-4e4cd09dc50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:24.408567 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efbefa4-7c9a-4f18-b4e9-4e4cd09dc50a admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:24.408856 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efbefa4-7c9a-4f18-b4e9-4e4cd09dc50a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:24.409136 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efbefa4-7c9a-4f18-b4e9-4e4cd09dc50a admin admin] PATH_INFO: `/v3/projects/a79aa014f11a4908bde66afd65873e80/users/6a21551e63014dfa98149712a5902123/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:24.424546 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efbefa4-7c9a-4f18-b4e9-4e4cd09dc50a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a79aa014f11a4908bde66afd65873e80, user_id=6a21551e63014dfa98149712a5902123, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:24.430358 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efbefa4-7c9a-4f18-b4e9-4e4cd09dc50a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:24.523877 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 891/1778] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:52:24 2022] PUT /identity/v3/projects/a79aa014f11a4908bde66afd65873e80/users/6a21551e63014dfa98149712a5902123/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:52:24.527659 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efbefa4-7c9a-4f18-b4e9-4e4cd09dc50a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:24.588089 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efbefa4-7c9a-4f18-b4e9-4e4cd09dc50a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:24.592892 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955aeb7-639d-4900-aca2-93d520d6f893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:24.593507 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955aeb7-639d-4900-aca2-93d520d6f893 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:24.593792 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955aeb7-639d-4900-aca2-93d520d6f893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:24.594097 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955aeb7-639d-4900-aca2-93d520d6f893 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:24.594814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1955aeb7-639d-4900-aca2-93d520d6f893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:24.596498 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1955aeb7-639d-4900-aca2-93d520d6f893 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:24.610892 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 892/1779] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:52:24.611560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955aeb7-639d-4900-aca2-93d520d6f893 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:24.623558 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955aeb7-639d-4900-aca2-93d520d6f893 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:24.630711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d8561-118f-4909-a3eb-b10ac8ce8a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:24.631650 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d8561-118f-4909-a3eb-b10ac8ce8a88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:24.631937 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d8561-118f-4909-a3eb-b10ac8ce8a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:24.632537 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d8561-118f-4909-a3eb-b10ac8ce8a88 admin admin] PATH_INFO: `/v3/projects/a79aa014f11a4908bde66afd65873e80/users/6a21551e63014dfa98149712a5902123/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:24.634969 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d8561-118f-4909-a3eb-b10ac8ce8a88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a79aa014f11a4908bde66afd65873e80, user_id=6a21551e63014dfa98149712a5902123, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:24.637625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d8561-118f-4909-a3eb-b10ac8ce8a88 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:24.640978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b84af62d-381e-4e7b-b11c-ba2c9f6aab15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a4d3f56e07e475c84a6910be016ff6a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:24.641728 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b84af62d-381e-4e7b-b11c-ba2c9f6aab15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a4d3f56e07e475c84a6910be016ff6a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:24.649322 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 893/1780] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:52:24 2022] PUT /identity/v3/projects/a79aa014f11a4908bde66afd65873e80/users/6a21551e63014dfa98149712a5902123/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:52:24.650411 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d8561-118f-4909-a3eb-b10ac8ce8a88 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:24.652097 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-692d8561-118f-4909-a3eb-b10ac8ce8a88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:24.666905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b84af62d-381e-4e7b-b11c-ba2c9f6aab15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a4d3f56e07e475c84a6910be016ff6a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:24.673959 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b84af62d-381e-4e7b-b11c-ba2c9f6aab15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a4d3f56e07e475c84a6910be016ff6a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:24.674703 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b84af62d-381e-4e7b-b11c-ba2c9f6aab15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a4d3f56e07e475c84a6910be016ff6a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:24.722375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d8561-118f-4909-a3eb-b10ac8ce8a88 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:24.732475 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ddb8b-0937-4bbe-9c14-9f701c3a66ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:24.732475 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ddb8b-0937-4bbe-9c14-9f701c3a66ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:24.732475 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ddb8b-0937-4bbe-9c14-9f701c3a66ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:24.732475 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ddb8b-0937-4bbe-9c14-9f701c3a66ef admin admin] PATH_INFO: `/v3/users/e418eb5ef9df406a94997aef7bbc8e02` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:24.733466 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ddb8b-0937-4bbe-9c14-9f701c3a66ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=e418eb5ef9df406a94997aef7bbc8e02)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:24.733466 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ddb8b-0937-4bbe-9c14-9f701c3a66ef admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:24.797183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 888/1781] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:52:24 2022] DELETE /identity/v3/users/7a4d3f56e07e475c84a6910be016ff6a => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:52:24.801318 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e9722-e5b6-4066-9eca-5c53107a1b85 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:24.801318 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e9722-e5b6-4066-9eca-5c53107a1b85 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:24.801318 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e9722-e5b6-4066-9eca-5c53107a1b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:24.831831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f71e9722-e5b6-4066-9eca-5c53107a1b85 None None] MFA Rules not processed for user `6a21551e63014dfa98149712a5902123`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:24.832727 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f71e9722-e5b6-4066-9eca-5c53107a1b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:24.919887 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 889/1782] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:24 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:24.931867 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71e9722-e5b6-4066-9eca-5c53107a1b85 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:24.933311 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f71e9722-e5b6-4066-9eca-5c53107a1b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:24.993132 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71e9722-e5b6-4066-9eca-5c53107a1b85 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:24.996887 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9689a2-bc5a-43cd-95c9-c1c687ce33d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:24.997511 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9689a2-bc5a-43cd-95c9-c1c687ce33d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:24.997798 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9689a2-bc5a-43cd-95c9-c1c687ce33d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:24.998079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9689a2-bc5a-43cd-95c9-c1c687ce33d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:25.009019 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9689a2-bc5a-43cd-95c9-c1c687ce33d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:25.010474 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9689a2-bc5a-43cd-95c9-c1c687ce33d8 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:25.022172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 890/1783] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:24 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:52:25.025667 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ab7ddb8b-0937-4bbe-9c14-9f701c3a66ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e418eb5ef9df406a94997aef7bbc8e02'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:25.026279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ab7ddb8b-0937-4bbe-9c14-9f701c3a66ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e418eb5ef9df406a94997aef7bbc8e02'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:25.050055 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ab7ddb8b-0937-4bbe-9c14-9f701c3a66ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e418eb5ef9df406a94997aef7bbc8e02'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:25.058745 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ab7ddb8b-0937-4bbe-9c14-9f701c3a66ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e418eb5ef9df406a94997aef7bbc8e02'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:25.059778 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ab7ddb8b-0937-4bbe-9c14-9f701c3a66ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e418eb5ef9df406a94997aef7bbc8e02'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:25.167959 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 894/1784] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:52:24 2022] DELETE /identity/v3/users/e418eb5ef9df406a94997aef7bbc8e02 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:25.171466 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41800089-1de4-4b87-8d10-b8566e7fab6f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:25.171777 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41800089-1de4-4b87-8d10-b8566e7fab6f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:25.172053 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41800089-1de4-4b87-8d10-b8566e7fab6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:25.192297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-41800089-1de4-4b87-8d10-b8566e7fab6f None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:25.193519 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-41800089-1de4-4b87-8d10-b8566e7fab6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:25.262625 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 891/1785] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:52:25.273239 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ddb8b-0937-4bbe-9c14-9f701c3a66ef admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:25.274731 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ab7ddb8b-0937-4bbe-9c14-9f701c3a66ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:25.336262 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ddb8b-0937-4bbe-9c14-9f701c3a66ef admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:25.339945 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5609687-c681-4821-a0b4-7723b39b4908 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:25.340572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5609687-c681-4821-a0b4-7723b39b4908 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:25.340860 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5609687-c681-4821-a0b4-7723b39b4908 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:25.341138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5609687-c681-4821-a0b4-7723b39b4908 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:25.350540 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5609687-c681-4821-a0b4-7723b39b4908 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:25.352473 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5609687-c681-4821-a0b4-7723b39b4908 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:25.360256 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41800089-1de4-4b87-8d10-b8566e7fab6f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:25.367805 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-41800089-1de4-4b87-8d10-b8566e7fab6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:25.368127 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 895/1786] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:52:25.375625 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41800089-1de4-4b87-8d10-b8566e7fab6f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:25.381033 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997c559-87ab-4b15-ae69-740f88dab8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:25.381987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997c559-87ab-4b15-ae69-740f88dab8c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:25.382432 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997c559-87ab-4b15-ae69-740f88dab8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:25.382871 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997c559-87ab-4b15-ae69-740f88dab8c7 admin admin] PATH_INFO: `/v3/projects/5a13b65a51be4d4eb058f172bdfc8ffe` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:25.384639 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6997c559-87ab-4b15-ae69-740f88dab8c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a13b65a51be4d4eb058f172bdfc8ffe)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:25.386565 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6997c559-87ab-4b15-ae69-740f88dab8c7 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:25.532765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6997c559-87ab-4b15-ae69-740f88dab8c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a13b65a51be4d4eb058f172bdfc8ffe', 'request_id': 'req-6997c559-87ab-4b15-ae69-740f88dab8c7'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:25.538863 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6997c559-87ab-4b15-ae69-740f88dab8c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a13b65a51be4d4eb058f172bdfc8ffe', 'request_id': 'req-6997c559-87ab-4b15-ae69-740f88dab8c7'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:25.677274 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6997c559-87ab-4b15-ae69-740f88dab8c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a13b65a51be4d4eb058f172bdfc8ffe', 'request_id': 'req-6997c559-87ab-4b15-ae69-740f88dab8c7'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:25.690181 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6997c559-87ab-4b15-ae69-740f88dab8c7 admin admin] The token cache is being invalidate because project 5a13b65a51be4d4eb058f172bdfc8ffe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:52:25.690474 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6997c559-87ab-4b15-ae69-740f88dab8c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:25.691146 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6997c559-87ab-4b15-ae69-740f88dab8c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:25.692990 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 892/1787] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:52:25 2022] DELETE /identity/v3/projects/5a13b65a51be4d4eb058f172bdfc8ffe => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:52:25.699317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9753c7-c6ba-42a7-b578-edbdb59e312e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:25.699665 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9753c7-c6ba-42a7-b578-edbdb59e312e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:25.699936 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9753c7-c6ba-42a7-b578-edbdb59e312e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:25.725660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-cd9753c7-c6ba-42a7-b578-edbdb59e312e None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:25.726996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-cd9753c7-c6ba-42a7-b578-edbdb59e312e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:25.804997 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 896/1788] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:25.813514 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba3619-81d0-4de7-a1de-0e9ad487b541 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:25.813807 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba3619-81d0-4de7-a1de-0e9ad487b541 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:25.814111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba3619-81d0-4de7-a1de-0e9ad487b541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:25.833528 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-caba3619-81d0-4de7-a1de-0e9ad487b541 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:25.834790 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-caba3619-81d0-4de7-a1de-0e9ad487b541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:25.855166 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 893/1789] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:25.855989 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9753c7-c6ba-42a7-b578-edbdb59e312e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:25.867531 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9753c7-c6ba-42a7-b578-edbdb59e312e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:25.872424 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0cf511-c44f-4024-bd6a-7fdf806eeec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:25.872967 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0cf511-c44f-4024-bd6a-7fdf806eeec4 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:25.873369 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0cf511-c44f-4024-bd6a-7fdf806eeec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:25.873643 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0cf511-c44f-4024-bd6a-7fdf806eeec4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:25.874404 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0cf511-c44f-4024-bd6a-7fdf806eeec4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:52:25.874991 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0cf511-c44f-4024-bd6a-7fdf806eeec4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:25.876847 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0cf511-c44f-4024-bd6a-7fdf806eeec4 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:25.891437 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 897/1790] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:52:25.896072 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba3619-81d0-4de7-a1de-0e9ad487b541 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:25.905792 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba3619-81d0-4de7-a1de-0e9ad487b541 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:25.917657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d394955-fb62-47f9-b3fe-f37517998e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:25.917657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d394955-fb62-47f9-b3fe-f37517998e54 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:25.917657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d394955-fb62-47f9-b3fe-f37517998e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:25.917657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d394955-fb62-47f9-b3fe-f37517998e54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:25.918022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d394955-fb62-47f9-b3fe-f37517998e54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:25.918022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d394955-fb62-47f9-b3fe-f37517998e54 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:25.947917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0cf511-c44f-4024-bd6a-7fdf806eeec4 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:25.950761 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-cf0cf511-c44f-4024-bd6a-7fdf806eeec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:25.971720 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0cf511-c44f-4024-bd6a-7fdf806eeec4 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:25.975909 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:26.000352 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77612c11-3c63-486f-92bc-a626eea758bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:26.001023 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77612c11-3c63-486f-92bc-a626eea758bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:26.001311 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77612c11-3c63-486f-92bc-a626eea758bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:26.001593 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77612c11-3c63-486f-92bc-a626eea758bb admin admin] PATH_INFO: `/v3/projects/0dbe90ceb0184024bc85bef3c688eaff` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:26.003051 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77612c11-3c63-486f-92bc-a626eea758bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dbe90ceb0184024bc85bef3c688eaff)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:26.005196 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77612c11-3c63-486f-92bc-a626eea758bb admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:26.068329 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 894/1791] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:52:25 2022] POST /identity/v3/projects => generated 367 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:52:26.074367 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d394955-fb62-47f9-b3fe-f37517998e54 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:26.138155 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d394955-fb62-47f9-b3fe-f37517998e54 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:26.145652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41f9a5-8678-4d63-8f1e-28aedd6fd724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:26.145652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41f9a5-8678-4d63-8f1e-28aedd6fd724 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:26.145652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41f9a5-8678-4d63-8f1e-28aedd6fd724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:26.145652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41f9a5-8678-4d63-8f1e-28aedd6fd724 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:26.145928 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f41f9a5-8678-4d63-8f1e-28aedd6fd724 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:26.148415 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f41f9a5-8678-4d63-8f1e-28aedd6fd724 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:26.194402 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-77612c11-3c63-486f-92bc-a626eea758bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dbe90ceb0184024bc85bef3c688eaff', 'request_id': 'req-77612c11-3c63-486f-92bc-a626eea758bb'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:26.200232 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-77612c11-3c63-486f-92bc-a626eea758bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dbe90ceb0184024bc85bef3c688eaff', 'request_id': 'req-77612c11-3c63-486f-92bc-a626eea758bb'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:26.237764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 895/1792] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:52:26 2022] POST /identity/v3/users => generated 284 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:52:26.241870 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41f9a5-8678-4d63-8f1e-28aedd6fd724 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:26.252221 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41f9a5-8678-4d63-8f1e-28aedd6fd724 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:26.294675 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-77612c11-3c63-486f-92bc-a626eea758bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dbe90ceb0184024bc85bef3c688eaff', 'request_id': 'req-77612c11-3c63-486f-92bc-a626eea758bb'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:26.308323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-77612c11-3c63-486f-92bc-a626eea758bb admin admin] The token cache is being invalidate because project 0dbe90ceb0184024bc85bef3c688eaff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:52:26.308666 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-77612c11-3c63-486f-92bc-a626eea758bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:26.310203 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-77612c11-3c63-486f-92bc-a626eea758bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:26.311901 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 898/1793] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:52:25 2022] DELETE /identity/v3/projects/0dbe90ceb0184024bc85bef3c688eaff => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:52:26.312947 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77612c11-3c63-486f-92bc-a626eea758bb admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:26.322488 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-77612c11-3c63-486f-92bc-a626eea758bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:26.323486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e7bca-ed5c-4ab1-8625-a0f003b7825e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:26.324693 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e7bca-ed5c-4ab1-8625-a0f003b7825e admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:26.324986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e7bca-ed5c-4ab1-8625-a0f003b7825e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:26.325267 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e7bca-ed5c-4ab1-8625-a0f003b7825e admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:26.327119 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e7bca-ed5c-4ab1-8625-a0f003b7825e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:26.329004 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e7bca-ed5c-4ab1-8625-a0f003b7825e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:26.336166 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77612c11-3c63-486f-92bc-a626eea758bb admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:26.344376 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 896/1794] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:52:26.344559 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5229f37c-e23b-4584-ae02-7edfea3e4971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:26.344559 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229f37c-e23b-4584-ae02-7edfea3e4971 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:26.344753 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80036ebe-5251-4179-9043-cb3083afa510 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:26.344832 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229f37c-e23b-4584-ae02-7edfea3e4971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:26.344832 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229f37c-e23b-4584-ae02-7edfea3e4971 admin admin] PATH_INFO: `/v3/users/122b9d2db6e549ffab33fbe7629754ce` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:26.345028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80036ebe-5251-4179-9043-cb3083afa510 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:26.345028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80036ebe-5251-4179-9043-cb3083afa510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:26.346503 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5229f37c-e23b-4584-ae02-7edfea3e4971 admin admin] RBAC: Authorizing `identity:delete_user(user_id=122b9d2db6e549ffab33fbe7629754ce)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:26.348705 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5229f37c-e23b-4584-ae02-7edfea3e4971 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:26.364800 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-80036ebe-5251-4179-9043-cb3083afa510 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:26.366807 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-80036ebe-5251-4179-9043-cb3083afa510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:26.371802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:26.378874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 897/1795] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:26.385408 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80036ebe-5251-4179-9043-cb3083afa510 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:26.385408 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-80036ebe-5251-4179-9043-cb3083afa510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:26.393215 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80036ebe-5251-4179-9043-cb3083afa510 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:26.398663 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a49fd-ba5b-446f-8e0d-3e7a4ed483ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:26.399469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a49fd-ba5b-446f-8e0d-3e7a4ed483ec admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:26.399760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a49fd-ba5b-446f-8e0d-3e7a4ed483ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:26.400039 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a49fd-ba5b-446f-8e0d-3e7a4ed483ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:26.400738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a49fd-ba5b-446f-8e0d-3e7a4ed483ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:26.402345 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a49fd-ba5b-446f-8e0d-3e7a4ed483ec admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:26.416334 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 898/1796] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:52:26.416334 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff658e-c4ba-43b8-b417-d5a629811b6e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:26.416334 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff658e-c4ba-43b8-b417-d5a629811b6e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:26.416334 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff658e-c4ba-43b8-b417-d5a629811b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:26.436476 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-9aff658e-c4ba-43b8-b417-d5a629811b6e None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:26.440276 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9aff658e-c4ba-43b8-b417-d5a629811b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:26.451600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 899/1797] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:26.453449 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff658e-c4ba-43b8-b417-d5a629811b6e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:26.463273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff658e-c4ba-43b8-b417-d5a629811b6e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:26.469419 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9c26b-8d4d-42d6-ba7c-ce6e7562b7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:26.470675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9c26b-8d4d-42d6-ba7c-ce6e7562b7d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:26.471120 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9c26b-8d4d-42d6-ba7c-ce6e7562b7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:26.471605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9c26b-8d4d-42d6-ba7c-ce6e7562b7d9 admin admin] PATH_INFO: `/v3/projects/8d066e1eae3e4e9c84261b9bfb988120/users/6890768d36fe4f34b661569372025201/roles/e033961a9363456a8510c34414dd839e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:26.495791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d9c26b-8d4d-42d6-ba7c-ce6e7562b7d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d066e1eae3e4e9c84261b9bfb988120, user_id=6890768d36fe4f34b661569372025201, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:26.499339 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d9c26b-8d4d-42d6-ba7c-ce6e7562b7d9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:26.711357 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-5229f37c-e23b-4584-ae02-7edfea3e4971 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '122b9d2db6e549ffab33fbe7629754ce'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:26.712054 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-5229f37c-e23b-4584-ae02-7edfea3e4971 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '122b9d2db6e549ffab33fbe7629754ce'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:26.728069 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-5229f37c-e23b-4584-ae02-7edfea3e4971 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '122b9d2db6e549ffab33fbe7629754ce'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:26.735067 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-5229f37c-e23b-4584-ae02-7edfea3e4971 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '122b9d2db6e549ffab33fbe7629754ce'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:26.735918 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-5229f37c-e23b-4584-ae02-7edfea3e4971 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '122b9d2db6e549ffab33fbe7629754ce'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:26.753222 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 900/1798] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:52:26 2022] PUT /identity/v3/projects/8d066e1eae3e4e9c84261b9bfb988120/users/6890768d36fe4f34b661569372025201/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:26.754350 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9c26b-8d4d-42d6-ba7c-ce6e7562b7d9 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:26.757329 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-43d9c26b-8d4d-42d6-ba7c-ce6e7562b7d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:26.825079 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 899/1799] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:52:26 2022] DELETE /identity/v3/users/122b9d2db6e549ffab33fbe7629754ce => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:26.827462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e16814-f627-4267-9e6f-10a514662b79 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:26.827462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e16814-f627-4267-9e6f-10a514662b79 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:26.827462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e16814-f627-4267-9e6f-10a514662b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:26.858075 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9c26b-8d4d-42d6-ba7c-ce6e7562b7d9 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:26.866580 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e7e16814-f627-4267-9e6f-10a514662b79 None None] MFA Rules not processed for user `6890768d36fe4f34b661569372025201`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:26.867891 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e7e16814-f627-4267-9e6f-10a514662b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:26.923240 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74b739-89fe-475b-abd7-6c6a74613e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:26.923912 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74b739-89fe-475b-abd7-6c6a74613e90 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:26.930650 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74b739-89fe-475b-abd7-6c6a74613e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:26.930650 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74b739-89fe-475b-abd7-6c6a74613e90 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:26.930650 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74b739-89fe-475b-abd7-6c6a74613e90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:52:26.930650 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74b739-89fe-475b-abd7-6c6a74613e90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:26.930650 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74b739-89fe-475b-abd7-6c6a74613e90 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:26.939873 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 900/1800] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:26 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:26.945019 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 901/1801] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:52:26.945176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e13211-58fa-46b1-8451-33854b83e8b7 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:26.945176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e13211-58fa-46b1-8451-33854b83e8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:26.945176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e13211-58fa-46b1-8451-33854b83e8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:26.945915 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74b739-89fe-475b-abd7-6c6a74613e90 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:26.955882 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74b739-89fe-475b-abd7-6c6a74613e90 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:26.966027 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b17f53-4876-456c-8d3e-a8f3e0e4b82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:26.966602 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e2e13211-58fa-46b1-8451-33854b83e8b7 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:26.966726 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b17f53-4876-456c-8d3e-a8f3e0e4b82e admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:26.966988 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b17f53-4876-456c-8d3e-a8f3e0e4b82e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:26.967879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e2e13211-58fa-46b1-8451-33854b83e8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:26.975698 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b17f53-4876-456c-8d3e-a8f3e0e4b82e admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:26.979014 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 901/1802] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:26.979150 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b17f53-4876-456c-8d3e-a8f3e0e4b82e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:26.979251 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e13211-58fa-46b1-8451-33854b83e8b7 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:26.979986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b17f53-4876-456c-8d3e-a8f3e0e4b82e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:26.985803 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e2e13211-58fa-46b1-8451-33854b83e8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:27.004570 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e13211-58fa-46b1-8451-33854b83e8b7 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:27.014702 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805caeb-be95-429a-b273-408da55d48c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:27.016625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805caeb-be95-429a-b273-408da55d48c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:27.016625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805caeb-be95-429a-b273-408da55d48c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:27.016625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805caeb-be95-429a-b273-408da55d48c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:27.016625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4805caeb-be95-429a-b273-408da55d48c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:27.018598 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4805caeb-be95-429a-b273-408da55d48c1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:27.132606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 902/1803] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:52:26 2022] POST /identity/v3/projects => generated 393 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:52:27.133752 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b17f53-4876-456c-8d3e-a8f3e0e4b82e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:27.137030 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-70b17f53-4876-456c-8d3e-a8f3e0e4b82e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:27.179475 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 902/1804] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:52:26 2022] POST /identity/v3/projects => generated 363 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:52:27.180585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805caeb-be95-429a-b273-408da55d48c1 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:27.204399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b17f53-4876-456c-8d3e-a8f3e0e4b82e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:27.252164 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805caeb-be95-429a-b273-408da55d48c1 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:27.257676 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e89a60-c225-4655-8f58-f4406fcddc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:27.258345 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e89a60-c225-4655-8f58-f4406fcddc2e admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:27.258907 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e89a60-c225-4655-8f58-f4406fcddc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:27.259354 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e89a60-c225-4655-8f58-f4406fcddc2e admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:27.260194 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e89a60-c225-4655-8f58-f4406fcddc2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:27.262290 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e89a60-c225-4655-8f58-f4406fcddc2e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:27.266601 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0964bf9-64ec-4bbc-b4c5-55168c14fe1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:27.267343 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0964bf9-64ec-4bbc-b4c5-55168c14fe1a service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:27.267692 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0964bf9-64ec-4bbc-b4c5-55168c14fe1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:27.267969 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0964bf9-64ec-4bbc-b4c5-55168c14fe1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:27.278869 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0964bf9-64ec-4bbc-b4c5-55168c14fe1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:27.280426 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0964bf9-64ec-4bbc-b4c5-55168c14fe1a service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:27.294366 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 903/1805] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:52:27.295536 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0964bf9-64ec-4bbc-b4c5-55168c14fe1a service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:27.307314 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0964bf9-64ec-4bbc-b4c5-55168c14fe1a service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:27.313695 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b451ec-a5a3-4263-8ca9-84f2dbe32d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:27.314676 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b451ec-a5a3-4263-8ca9-84f2dbe32d67 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:27.316221 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b451ec-a5a3-4263-8ca9-84f2dbe32d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:27.316221 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b451ec-a5a3-4263-8ca9-84f2dbe32d67 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:27.316221 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b451ec-a5a3-4263-8ca9-84f2dbe32d67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:27.317929 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b451ec-a5a3-4263-8ca9-84f2dbe32d67 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:27.369974 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 903/1806] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:52:27 2022] POST /identity/v3/users => generated 297 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:52:27.375721 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e89a60-c225-4655-8f58-f4406fcddc2e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:27.386813 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e89a60-c225-4655-8f58-f4406fcddc2e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:27.391949 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738851d-defc-402e-92d8-73d73b3d3c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:27.392571 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738851d-defc-402e-92d8-73d73b3d3c2a admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:27.392851 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738851d-defc-402e-92d8-73d73b3d3c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:27.393130 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738851d-defc-402e-92d8-73d73b3d3c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:27.393836 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738851d-defc-402e-92d8-73d73b3d3c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:27.395509 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738851d-defc-402e-92d8-73d73b3d3c2a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:27.406444 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 904/1807] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:52:27.410626 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738851d-defc-402e-92d8-73d73b3d3c2a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:27.420083 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738851d-defc-402e-92d8-73d73b3d3c2a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:27.424963 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be627b5-66c1-4689-be96-87f9cb8fd78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:27.425671 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be627b5-66c1-4689-be96-87f9cb8fd78e admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:27.425955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be627b5-66c1-4689-be96-87f9cb8fd78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:27.426234 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be627b5-66c1-4689-be96-87f9cb8fd78e admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:27.426910 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be627b5-66c1-4689-be96-87f9cb8fd78e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:27.428534 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be627b5-66c1-4689-be96-87f9cb8fd78e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:27.433737 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 904/1808] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:52:27 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:52:27.439122 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b451ec-a5a3-4263-8ca9-84f2dbe32d67 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:27.439310 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 905/1809] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:52:27.443448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be627b5-66c1-4689-be96-87f9cb8fd78e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:27.449545 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b451ec-a5a3-4263-8ca9-84f2dbe32d67 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:27.453751 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be627b5-66c1-4689-be96-87f9cb8fd78e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:27.454569 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b83ef-65ad-4c26-b1d2-9bcd64145af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:27.454941 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b83ef-65ad-4c26-b1d2-9bcd64145af6 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:27.455227 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b83ef-65ad-4c26-b1d2-9bcd64145af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:27.455549 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b83ef-65ad-4c26-b1d2-9bcd64145af6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:27.456267 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95b83ef-65ad-4c26-b1d2-9bcd64145af6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:27.457903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95b83ef-65ad-4c26-b1d2-9bcd64145af6 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:27.458769 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b783e7d7-23ac-4fcf-aa4d-9a0e4a98d48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:27.459574 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783e7d7-23ac-4fcf-aa4d-9a0e4a98d48b admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:27.460434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783e7d7-23ac-4fcf-aa4d-9a0e4a98d48b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:27.460721 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783e7d7-23ac-4fcf-aa4d-9a0e4a98d48b admin admin] PATH_INFO: `/v3/projects/9018edf54c1c4a2197591cc99a814bde/users/5b98f1449be74722ad14ca253365988b/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:27.470606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 905/1810] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:52:27.479677 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b83ef-65ad-4c26-b1d2-9bcd64145af6 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:27.480760 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b783e7d7-23ac-4fcf-aa4d-9a0e4a98d48b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9018edf54c1c4a2197591cc99a814bde, user_id=5b98f1449be74722ad14ca253365988b, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:27.484852 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b783e7d7-23ac-4fcf-aa4d-9a0e4a98d48b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:27.489634 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b83ef-65ad-4c26-b1d2-9bcd64145af6 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:27.494246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecac36-f3eb-4674-91c1-41cf522ea73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:27.497207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecac36-f3eb-4674-91c1-41cf522ea73f admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:27.497207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecac36-f3eb-4674-91c1-41cf522ea73f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:27.497207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecac36-f3eb-4674-91c1-41cf522ea73f admin admin] PATH_INFO: `/v3/projects/4180d8b6c8b5493f9843effef37373a3/users/536710bbb8014a708c6d88fc25d28cd8/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:27.510944 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ecac36-f3eb-4674-91c1-41cf522ea73f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4180d8b6c8b5493f9843effef37373a3, user_id=536710bbb8014a708c6d88fc25d28cd8, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:27.513680 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ecac36-f3eb-4674-91c1-41cf522ea73f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:27.627983 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 906/1811] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:52:27 2022] PUT /identity/v3/projects/9018edf54c1c4a2197591cc99a814bde/users/5b98f1449be74722ad14ca253365988b/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:52:27.638496 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa384958-40a1-4275-8194-c76606608c60 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:27.638890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa384958-40a1-4275-8194-c76606608c60 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:27.639246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa384958-40a1-4275-8194-c76606608c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:27.672761 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 906/1812] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:52:27 2022] PUT /identity/v3/projects/4180d8b6c8b5493f9843effef37373a3/users/536710bbb8014a708c6d88fc25d28cd8/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:52:27.676978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecac36-f3eb-4674-91c1-41cf522ea73f admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:27.700730 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-aa384958-40a1-4275-8194-c76606608c60 None None] MFA Rules not processed for user `5b98f1449be74722ad14ca253365988b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:27.701968 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-aa384958-40a1-4275-8194-c76606608c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:27.761698 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecac36-f3eb-4674-91c1-41cf522ea73f admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:27.766608 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8564265d-dbd1-4a84-ba3c-406ea370495c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:27.767257 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564265d-dbd1-4a84-ba3c-406ea370495c admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:27.767636 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564265d-dbd1-4a84-ba3c-406ea370495c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:27.767919 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564265d-dbd1-4a84-ba3c-406ea370495c admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:27.768620 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8564265d-dbd1-4a84-ba3c-406ea370495c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:27.770356 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8564265d-dbd1-4a84-ba3c-406ea370495c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:27.776842 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 907/1813] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:27.777555 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa384958-40a1-4275-8194-c76606608c60 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:27.783450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 907/1814] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:52:27.783450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8564265d-dbd1-4a84-ba3c-406ea370495c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:27.784887 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8564265d-dbd1-4a84-ba3c-406ea370495c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:27.793539 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa384958-40a1-4275-8194-c76606608c60 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:27.798297 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee46d1b-96cc-43b3-8352-83e519aa458a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:27.798922 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee46d1b-96cc-43b3-8352-83e519aa458a admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:27.799820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee46d1b-96cc-43b3-8352-83e519aa458a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:27.800094 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee46d1b-96cc-43b3-8352-83e519aa458a admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:27.800788 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee46d1b-96cc-43b3-8352-83e519aa458a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:27.800943 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8564265d-dbd1-4a84-ba3c-406ea370495c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:27.803461 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee46d1b-96cc-43b3-8352-83e519aa458a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:27.808048 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a676ae-d1e6-4766-a6a0-77d1b096ee89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:27.808855 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a676ae-d1e6-4766-a6a0-77d1b096ee89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:27.809017 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a676ae-d1e6-4766-a6a0-77d1b096ee89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:27.809284 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a676ae-d1e6-4766-a6a0-77d1b096ee89 admin admin] PATH_INFO: `/v3/projects/2e3803254b504f59bd5c20ffbceeeede` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:27.810651 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a676ae-d1e6-4766-a6a0-77d1b096ee89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e3803254b504f59bd5c20ffbceeeede)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:27.812708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a676ae-d1e6-4766-a6a0-77d1b096ee89 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:27.908799 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 908/1815] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:52:27 2022] POST /identity/v3/projects => generated 393 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:52:27.910019 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee46d1b-96cc-43b3-8352-83e519aa458a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:27.950557 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e1a676ae-d1e6-4766-a6a0-77d1b096ee89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e3803254b504f59bd5c20ffbceeeede', 'request_id': 'req-e1a676ae-d1e6-4766-a6a0-77d1b096ee89'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:27.957434 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e1a676ae-d1e6-4766-a6a0-77d1b096ee89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e3803254b504f59bd5c20ffbceeeede', 'request_id': 'req-e1a676ae-d1e6-4766-a6a0-77d1b096ee89'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:27.979290 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee46d1b-96cc-43b3-8352-83e519aa458a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:28.041596 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45124ef2-4c17-44f8-a546-895ab6b1b435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:28.047303 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45124ef2-4c17-44f8-a546-895ab6b1b435 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:28.047303 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45124ef2-4c17-44f8-a546-895ab6b1b435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:28.047303 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45124ef2-4c17-44f8-a546-895ab6b1b435 admin admin] PATH_INFO: `/v3/projects/4180d8b6c8b5493f9843effef37373a3/users/536710bbb8014a708c6d88fc25d28cd8/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:28.047303 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45124ef2-4c17-44f8-a546-895ab6b1b435 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4180d8b6c8b5493f9843effef37373a3, user_id=536710bbb8014a708c6d88fc25d28cd8, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:28.047883 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45124ef2-4c17-44f8-a546-895ab6b1b435 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:28.060766 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 909/1816] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:52:27 2022] PUT /identity/v3/projects/4180d8b6c8b5493f9843effef37373a3/users/536710bbb8014a708c6d88fc25d28cd8/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:28.061588 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45124ef2-4c17-44f8-a546-895ab6b1b435 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:28.072207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e1a676ae-d1e6-4766-a6a0-77d1b096ee89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e3803254b504f59bd5c20ffbceeeede', 'request_id': 'req-e1a676ae-d1e6-4766-a6a0-77d1b096ee89'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:28.086209 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e1a676ae-d1e6-4766-a6a0-77d1b096ee89 admin admin] The token cache is being invalidate because project 2e3803254b504f59bd5c20ffbceeeede was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:52:28.086542 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e1a676ae-d1e6-4766-a6a0-77d1b096ee89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:28.087171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e1a676ae-d1e6-4766-a6a0-77d1b096ee89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:28.089141 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 908/1817] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:52:27 2022] DELETE /identity/v3/projects/2e3803254b504f59bd5c20ffbceeeede => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:28.090537 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754936b-f2bb-45dd-8a74-9598f9cd7b8e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:28.090816 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754936b-f2bb-45dd-8a74-9598f9cd7b8e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:28.091086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754936b-f2bb-45dd-8a74-9598f9cd7b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:28.119930 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f754936b-f2bb-45dd-8a74-9598f9cd7b8e None None] MFA Rules not processed for user `536710bbb8014a708c6d88fc25d28cd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:28.121149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f754936b-f2bb-45dd-8a74-9598f9cd7b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:28.137841 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45124ef2-4c17-44f8-a546-895ab6b1b435 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:28.142661 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd6d60-a209-4097-9732-ae0386f2eb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:28.143283 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd6d60-a209-4097-9732-ae0386f2eb2b admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:28.143599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd6d60-a209-4097-9732-ae0386f2eb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:28.143879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd6d60-a209-4097-9732-ae0386f2eb2b admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:28.144543 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cd6d60-a209-4097-9732-ae0386f2eb2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:28.146474 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cd6d60-a209-4097-9732-ae0386f2eb2b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:28.193933 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 909/1818] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:28 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:28.195583 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754936b-f2bb-45dd-8a74-9598f9cd7b8e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:28.197192 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f754936b-f2bb-45dd-8a74-9598f9cd7b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:28.212374 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754936b-f2bb-45dd-8a74-9598f9cd7b8e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:28.220569 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d61a8c-fe84-4ccc-a130-e81cf47b3eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:28.221236 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d61a8c-fe84-4ccc-a130-e81cf47b3eff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:28.221525 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d61a8c-fe84-4ccc-a130-e81cf47b3eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:28.221870 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d61a8c-fe84-4ccc-a130-e81cf47b3eff admin admin] PATH_INFO: `/v3/users/6a21551e63014dfa98149712a5902123` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:28.223764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d61a8c-fe84-4ccc-a130-e81cf47b3eff admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a21551e63014dfa98149712a5902123)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:28.226342 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d61a8c-fe84-4ccc-a130-e81cf47b3eff admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:28.285922 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 910/1819] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:52:28 2022] POST /identity/v3/users => generated 303 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:52:28.285922 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd6d60-a209-4097-9732-ae0386f2eb2b admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:28.287609 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f6cd6d60-a209-4097-9732-ae0386f2eb2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:28.347355 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd6d60-a209-4097-9732-ae0386f2eb2b admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:28.351648 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e2d20-0d3c-4dfb-8e9b-c843b85adc70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:28.352272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e2d20-0d3c-4dfb-8e9b-c843b85adc70 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:28.352555 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e2d20-0d3c-4dfb-8e9b-c843b85adc70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:28.352832 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e2d20-0d3c-4dfb-8e9b-c843b85adc70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:28.362237 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97e2d20-0d3c-4dfb-8e9b-c843b85adc70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:28.363704 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97e2d20-0d3c-4dfb-8e9b-c843b85adc70 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:28.377946 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 911/1820] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:28 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:52:28.377946 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e2d20-0d3c-4dfb-8e9b-c843b85adc70 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:28.377946 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e97e2d20-0d3c-4dfb-8e9b-c843b85adc70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:28.391571 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e2d20-0d3c-4dfb-8e9b-c843b85adc70 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:28.396482 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c81ccd-6a72-4840-8e25-2ec54bfa0c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:28.397108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c81ccd-6a72-4840-8e25-2ec54bfa0c68 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:28.397954 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c81ccd-6a72-4840-8e25-2ec54bfa0c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:28.398236 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c81ccd-6a72-4840-8e25-2ec54bfa0c68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:28.398925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c81ccd-6a72-4840-8e25-2ec54bfa0c68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:28.400753 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c81ccd-6a72-4840-8e25-2ec54bfa0c68 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:28.411558 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 912/1821] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:52:28.415657 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c81ccd-6a72-4840-8e25-2ec54bfa0c68 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:28.425596 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c81ccd-6a72-4840-8e25-2ec54bfa0c68 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:28.430763 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287b671-1894-4445-8639-8546fedf20fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:28.431480 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287b671-1894-4445-8639-8546fedf20fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:28.431768 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287b671-1894-4445-8639-8546fedf20fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:28.432049 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287b671-1894-4445-8639-8546fedf20fb admin admin] PATH_INFO: `/v3/projects/8534fae5fc9a487cb6b298cd03b2146f/users/4edf96e86ce249b283265424c7880c40/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:28.447368 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8287b671-1894-4445-8639-8546fedf20fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8534fae5fc9a487cb6b298cd03b2146f, user_id=4edf96e86ce249b283265424c7880c40, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:28.450140 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8287b671-1894-4445-8639-8546fedf20fb admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:28.502506 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-67d61a8c-fe84-4ccc-a130-e81cf47b3eff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a21551e63014dfa98149712a5902123'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:28.504073 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-67d61a8c-fe84-4ccc-a130-e81cf47b3eff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a21551e63014dfa98149712a5902123'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:28.520761 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-67d61a8c-fe84-4ccc-a130-e81cf47b3eff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a21551e63014dfa98149712a5902123'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:28.530640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-67d61a8c-fe84-4ccc-a130-e81cf47b3eff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a21551e63014dfa98149712a5902123'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:28.531536 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-67d61a8c-fe84-4ccc-a130-e81cf47b3eff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a21551e63014dfa98149712a5902123'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:28.587300 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 913/1822] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:52:28 2022] PUT /identity/v3/projects/8534fae5fc9a487cb6b298cd03b2146f/users/4edf96e86ce249b283265424c7880c40/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:52:28.591196 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287b671-1894-4445-8639-8546fedf20fb admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:28.593159 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8287b671-1894-4445-8639-8546fedf20fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:28.655596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 910/1823] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:52:28 2022] DELETE /identity/v3/users/6a21551e63014dfa98149712a5902123 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:28.656444 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287b671-1894-4445-8639-8546fedf20fb admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:28.670826 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d61a8c-fe84-4ccc-a130-e81cf47b3eff admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:28.672113 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-67d61a8c-fe84-4ccc-a130-e81cf47b3eff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:28.729276 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6f68d-324b-44fc-8db7-a418bd283615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:28.730038 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6f68d-324b-44fc-8db7-a418bd283615 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:28.730402 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6f68d-324b-44fc-8db7-a418bd283615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:28.732087 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6f68d-324b-44fc-8db7-a418bd283615 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:28.732087 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b6f68d-324b-44fc-8db7-a418bd283615 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:28.733522 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b6f68d-324b-44fc-8db7-a418bd283615 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:28.741816 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d61a8c-fe84-4ccc-a130-e81cf47b3eff admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:28.746333 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb17365-7ff2-4c67-a9f5-b7e2046e8ac3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:28.747036 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb17365-7ff2-4c67-a9f5-b7e2046e8ac3 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:28.747322 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb17365-7ff2-4c67-a9f5-b7e2046e8ac3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:28.747645 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb17365-7ff2-4c67-a9f5-b7e2046e8ac3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:28.749909 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 914/1824] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:52:28.750697 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fbb17365-7ff2-4c67-a9f5-b7e2046e8ac3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:28.751198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6f68d-324b-44fc-8db7-a418bd283615 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:28.763659 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6f68d-324b-44fc-8db7-a418bd283615 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:28.769041 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d52e5-28f3-4351-baae-d1cdbf9eea33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:28.769781 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d52e5-28f3-4351-baae-d1cdbf9eea33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:28.770067 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d52e5-28f3-4351-baae-d1cdbf9eea33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:28.770346 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d52e5-28f3-4351-baae-d1cdbf9eea33 admin admin] PATH_INFO: `/v3/projects/8534fae5fc9a487cb6b298cd03b2146f/users/4edf96e86ce249b283265424c7880c40/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:28.772606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d52e5-28f3-4351-baae-d1cdbf9eea33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8534fae5fc9a487cb6b298cd03b2146f, user_id=4edf96e86ce249b283265424c7880c40, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:28.775239 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d52e5-28f3-4351-baae-d1cdbf9eea33 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:28.791430 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 915/1825] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:52:28 2022] PUT /identity/v3/projects/8534fae5fc9a487cb6b298cd03b2146f/users/4edf96e86ce249b283265424c7880c40/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:28.802855 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2ea61-143e-4dd2-80fb-a76b05f3247e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:28.803321 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2ea61-143e-4dd2-80fb-a76b05f3247e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:28.803823 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2ea61-143e-4dd2-80fb-a76b05f3247e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:28.834186 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb17365-7ff2-4c67-a9f5-b7e2046e8ac3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:28.837950 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3ee2ea61-143e-4dd2-80fb-a76b05f3247e None None] MFA Rules not processed for user `4edf96e86ce249b283265424c7880c40`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:28.838071 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb17365-7ff2-4c67-a9f5-b7e2046e8ac3 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:28.838176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3ee2ea61-143e-4dd2-80fb-a76b05f3247e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:28.847580 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 911/1826] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:28 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:52:28.879501 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:28.940987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 916/1827] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:28.953157 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb17365-7ff2-4c67-a9f5-b7e2046e8ac3 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:28.955041 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fbb17365-7ff2-4c67-a9f5-b7e2046e8ac3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:28.966370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb17365-7ff2-4c67-a9f5-b7e2046e8ac3 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:29.029887 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d363fd-e744-4ca1-a945-b82516f5c0b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:29.029887 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d363fd-e744-4ca1-a945-b82516f5c0b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:29.029887 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d363fd-e744-4ca1-a945-b82516f5c0b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:29.029887 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d363fd-e744-4ca1-a945-b82516f5c0b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:29.042646 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d363fd-e744-4ca1-a945-b82516f5c0b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:29.042646 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d363fd-e744-4ca1-a945-b82516f5c0b8 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:29.051745 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 912/1828] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:28 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:52:29.128422 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2ea61-143e-4dd2-80fb-a76b05f3247e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:29.134074 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2ea61-143e-4dd2-80fb-a76b05f3247e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:29.138993 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dafa5b-4bce-43c8-8c14-f130b3d00154 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:29.138993 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dafa5b-4bce-43c8-8c14-f130b3d00154 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:29.138993 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dafa5b-4bce-43c8-8c14-f130b3d00154 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:29.138993 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dafa5b-4bce-43c8-8c14-f130b3d00154 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:29.140453 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c6dafa5b-4bce-43c8-8c14-f130b3d00154 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:29.209042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d363fd-e744-4ca1-a945-b82516f5c0b8 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:29.211421 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d6d363fd-e744-4ca1-a945-b82516f5c0b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:29.217465 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dafa5b-4bce-43c8-8c14-f130b3d00154 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:29.219072 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dafa5b-4bce-43c8-8c14-f130b3d00154 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:29.229858 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 917/1829] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:52:29.297586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d363fd-e744-4ca1-a945-b82516f5c0b8 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:29.304574 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4316fb-9f05-432a-b9f1-47d6479ead13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:29.305237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4316fb-9f05-432a-b9f1-47d6479ead13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:29.305521 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4316fb-9f05-432a-b9f1-47d6479ead13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:29.305897 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4316fb-9f05-432a-b9f1-47d6479ead13 admin admin] PATH_INFO: `/v3/projects/a79aa014f11a4908bde66afd65873e80` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:29.306976 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4316fb-9f05-432a-b9f1-47d6479ead13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a79aa014f11a4908bde66afd65873e80)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:29.308954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4316fb-9f05-432a-b9f1-47d6479ead13 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:29.312915 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:29.417413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7a4316fb-9f05-432a-b9f1-47d6479ead13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a79aa014f11a4908bde66afd65873e80', 'request_id': 'req-7a4316fb-9f05-432a-b9f1-47d6479ead13'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:29.423004 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7a4316fb-9f05-432a-b9f1-47d6479ead13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a79aa014f11a4908bde66afd65873e80', 'request_id': 'req-7a4316fb-9f05-432a-b9f1-47d6479ead13'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:29.541086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7a4316fb-9f05-432a-b9f1-47d6479ead13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a79aa014f11a4908bde66afd65873e80', 'request_id': 'req-7a4316fb-9f05-432a-b9f1-47d6479ead13'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:29.556854 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7a4316fb-9f05-432a-b9f1-47d6479ead13 admin admin] The token cache is being invalidate because project a79aa014f11a4908bde66afd65873e80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:52:29.557182 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7a4316fb-9f05-432a-b9f1-47d6479ead13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:29.558091 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7a4316fb-9f05-432a-b9f1-47d6479ead13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:29.559882 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 913/1830] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:52:29 2022] DELETE /identity/v3/projects/a79aa014f11a4908bde66afd65873e80 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:29.564743 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04372133-fdf3-425a-96e5-fab8b641bd98 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:29.565193 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04372133-fdf3-425a-96e5-fab8b641bd98 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:29.565313 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04372133-fdf3-425a-96e5-fab8b641bd98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:29.584631 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-04372133-fdf3-425a-96e5-fab8b641bd98 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:29.585827 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-04372133-fdf3-425a-96e5-fab8b641bd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:29.647256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 918/1831] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:29.655307 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a337fef-cc95-4290-9412-8664b275fcdf None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:29.655631 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a337fef-cc95-4290-9412-8664b275fcdf None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:29.655919 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a337fef-cc95-4290-9412-8664b275fcdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:29.675991 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6a337fef-cc95-4290-9412-8664b275fcdf None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:29.677243 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6a337fef-cc95-4290-9412-8664b275fcdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:29.685314 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 914/1832] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:29.689585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04372133-fdf3-425a-96e5-fab8b641bd98 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:29.700772 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04372133-fdf3-425a-96e5-fab8b641bd98 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:29.709454 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20589592-a6af-4018-8a46-939e3455549d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:29.709454 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20589592-a6af-4018-8a46-939e3455549d admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:29.709454 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20589592-a6af-4018-8a46-939e3455549d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:29.709454 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20589592-a6af-4018-8a46-939e3455549d admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:29.709785 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20589592-a6af-4018-8a46-939e3455549d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:52:29.709785 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20589592-a6af-4018-8a46-939e3455549d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:29.711933 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20589592-a6af-4018-8a46-939e3455549d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:29.727613 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 919/1833] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:52:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:52:29.729213 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a337fef-cc95-4290-9412-8664b275fcdf None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:29.738744 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a337fef-cc95-4290-9412-8664b275fcdf None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:29.743958 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c20a3-4175-487a-98c1-8af6307486dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:29.744739 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c20a3-4175-487a-98c1-8af6307486dd admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:29.744890 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c20a3-4175-487a-98c1-8af6307486dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:29.745159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c20a3-4175-487a-98c1-8af6307486dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:29.745799 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c20a3-4175-487a-98c1-8af6307486dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:29.747802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c20a3-4175-487a-98c1-8af6307486dd admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:29.895614 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 915/1834] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:52:29 2022] POST /identity/v3/projects => generated 355 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:52:29.898157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20589592-a6af-4018-8a46-939e3455549d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:29.917871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20589592-a6af-4018-8a46-939e3455549d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:30.028188 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a246badc-0007-490f-b5bd-6d30de247b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:30.028188 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246badc-0007-490f-b5bd-6d30de247b80 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:30.028188 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246badc-0007-490f-b5bd-6d30de247b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:30.028188 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246badc-0007-490f-b5bd-6d30de247b80 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:30.028506 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a246badc-0007-490f-b5bd-6d30de247b80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:30.028506 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a246badc-0007-490f-b5bd-6d30de247b80 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:30.202144 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 920/1835] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:52:29 2022] POST /identity/v3/users => generated 278 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:52:30.203455 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c20a3-4175-487a-98c1-8af6307486dd admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:30.214527 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c20a3-4175-487a-98c1-8af6307486dd admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:30.219549 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a81176-cefb-4237-8a2f-f63658680eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:30.220252 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a81176-cefb-4237-8a2f-f63658680eb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:30.220609 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a81176-cefb-4237-8a2f-f63658680eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:30.220992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a81176-cefb-4237-8a2f-f63658680eb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:30.221795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a81176-cefb-4237-8a2f-f63658680eb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:30.223578 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a81176-cefb-4237-8a2f-f63658680eb3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:30.236076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 916/1836] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:52:30.239333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a246badc-0007-490f-b5bd-6d30de247b80 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:30.251410 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a246badc-0007-490f-b5bd-6d30de247b80 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:30.258422 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5369a20e-7b8d-4067-9c0b-f1060d6cecb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:30.259271 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369a20e-7b8d-4067-9c0b-f1060d6cecb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:30.259702 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369a20e-7b8d-4067-9c0b-f1060d6cecb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:30.260118 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369a20e-7b8d-4067-9c0b-f1060d6cecb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:30.260979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5369a20e-7b8d-4067-9c0b-f1060d6cecb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:30.262942 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5369a20e-7b8d-4067-9c0b-f1060d6cecb0 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:30.283491 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 921/1837] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:52:30.286724 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a81176-cefb-4237-8a2f-f63658680eb3 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:30.297557 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a81176-cefb-4237-8a2f-f63658680eb3 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:30.303533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb6898-06d4-4c92-b1de-76d4e30eb8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:30.304376 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb6898-06d4-4c92-b1de-76d4e30eb8df admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:30.304741 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb6898-06d4-4c92-b1de-76d4e30eb8df admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:30.305030 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb6898-06d4-4c92-b1de-76d4e30eb8df admin admin] PATH_INFO: `/v3/projects/03b667b40abe4a52969ce1787c1c0189/users/fac59206b5b54053be9a28970097b8aa/roles/e033961a9363456a8510c34414dd839e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:30.321353 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb6898-06d4-4c92-b1de-76d4e30eb8df admin admin] RBAC: Authorizing `identity:create_grant(project_id=03b667b40abe4a52969ce1787c1c0189, user_id=fac59206b5b54053be9a28970097b8aa, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:30.324391 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb6898-06d4-4c92-b1de-76d4e30eb8df admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:30.437841 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 917/1838] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:52:30 2022] PUT /identity/v3/projects/03b667b40abe4a52969ce1787c1c0189/users/fac59206b5b54053be9a28970097b8aa/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:30.448412 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc78566-5e50-4de6-ac96-15a3551e0a05 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:30.448704 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc78566-5e50-4de6-ac96-15a3551e0a05 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:30.448987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc78566-5e50-4de6-ac96-15a3551e0a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:30.476706 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-0cc78566-5e50-4de6-ac96-15a3551e0a05 None None] MFA Rules not processed for user `fac59206b5b54053be9a28970097b8aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:30.477938 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0cc78566-5e50-4de6-ac96-15a3551e0a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:30.548326 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 922/1839] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:30 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:30.549348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb6898-06d4-4c92-b1de-76d4e30eb8df admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:30.613357 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb6898-06d4-4c92-b1de-76d4e30eb8df admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:30.618964 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8579b3b-6e67-47c2-a006-e9e255170fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:30.619287 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8579b3b-6e67-47c2-a006-e9e255170fd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:30.619604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8579b3b-6e67-47c2-a006-e9e255170fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:30.619882 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8579b3b-6e67-47c2-a006-e9e255170fd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:30.620564 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8579b3b-6e67-47c2-a006-e9e255170fd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:30.622460 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8579b3b-6e67-47c2-a006-e9e255170fd1 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:30.737523 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 918/1840] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:52:30 2022] POST /identity/v3/projects => generated 357 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:52:30.742331 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc78566-5e50-4de6-ac96-15a3551e0a05 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:30.753212 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc78566-5e50-4de6-ac96-15a3551e0a05 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:30.810514 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7596c70-6884-44cf-8d23-5eb3076efb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:30.811149 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7596c70-6884-44cf-8d23-5eb3076efb48 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:30.811459 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7596c70-6884-44cf-8d23-5eb3076efb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:30.811738 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7596c70-6884-44cf-8d23-5eb3076efb48 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:30.812400 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7596c70-6884-44cf-8d23-5eb3076efb48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:30.814395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7596c70-6884-44cf-8d23-5eb3076efb48 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:30.932008 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 923/1841] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:52:30 2022] POST /identity/v3/users => generated 285 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:52:30.935983 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8579b3b-6e67-47c2-a006-e9e255170fd1 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:30.946631 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8579b3b-6e67-47c2-a006-e9e255170fd1 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:30.951353 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4561f4d-a008-41a7-8c72-cbbe9fbf52bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:30.952116 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4561f4d-a008-41a7-8c72-cbbe9fbf52bd admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:30.952405 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4561f4d-a008-41a7-8c72-cbbe9fbf52bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:30.952685 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4561f4d-a008-41a7-8c72-cbbe9fbf52bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:30.953374 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4561f4d-a008-41a7-8c72-cbbe9fbf52bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:30.954980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4561f4d-a008-41a7-8c72-cbbe9fbf52bd admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:30.965420 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 919/1842] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:52:30.969555 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7596c70-6884-44cf-8d23-5eb3076efb48 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:30.979740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7596c70-6884-44cf-8d23-5eb3076efb48 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:30.984416 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7693c012-581f-4e13-912a-9711658e68a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:30.986549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693c012-581f-4e13-912a-9711658e68a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:30.986549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693c012-581f-4e13-912a-9711658e68a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:30.986549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693c012-581f-4e13-912a-9711658e68a1 admin admin] PATH_INFO: `/v3/projects/6a4b82827112444394e3808b69b20573/users/5dd91094a71448b8a6bf44be2c2b1d5b/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:31.000661 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7693c012-581f-4e13-912a-9711658e68a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a4b82827112444394e3808b69b20573, user_id=5dd91094a71448b8a6bf44be2c2b1d5b, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:31.003331 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7693c012-581f-4e13-912a-9711658e68a1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:31.106939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 924/1843] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:52:30 2022] PUT /identity/v3/projects/6a4b82827112444394e3808b69b20573/users/5dd91094a71448b8a6bf44be2c2b1d5b/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:31.108204 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4561f4d-a008-41a7-8c72-cbbe9fbf52bd admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:31.119901 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4561f4d-a008-41a7-8c72-cbbe9fbf52bd admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:31.199441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ad4da-aefb-4e1f-8b14-89746b1c34a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:31.200076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ad4da-aefb-4e1f-8b14-89746b1c34a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:31.200364 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ad4da-aefb-4e1f-8b14-89746b1c34a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:31.200638 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ad4da-aefb-4e1f-8b14-89746b1c34a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:31.201339 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8ad4da-aefb-4e1f-8b14-89746b1c34a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:31.202913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8ad4da-aefb-4e1f-8b14-89746b1c34a1 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:31.218256 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 920/1844] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:31 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:52:31.223431 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7693c012-581f-4e13-912a-9711658e68a1 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:31.238399 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7693c012-581f-4e13-912a-9711658e68a1 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:31.243291 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852da61f-fc1a-47b3-8ebe-8f7362685fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:31.244014 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852da61f-fc1a-47b3-8ebe-8f7362685fac admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:31.244504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852da61f-fc1a-47b3-8ebe-8f7362685fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:31.244606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852da61f-fc1a-47b3-8ebe-8f7362685fac admin admin] PATH_INFO: `/v3/projects/6a4b82827112444394e3808b69b20573/users/5dd91094a71448b8a6bf44be2c2b1d5b/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:31.246663 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852da61f-fc1a-47b3-8ebe-8f7362685fac admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a4b82827112444394e3808b69b20573, user_id=5dd91094a71448b8a6bf44be2c2b1d5b, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:31.249510 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852da61f-fc1a-47b3-8ebe-8f7362685fac admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:31.268694 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 925/1845] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:52:31 2022] PUT /identity/v3/projects/6a4b82827112444394e3808b69b20573/users/5dd91094a71448b8a6bf44be2c2b1d5b/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:31.273564 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214c634-1051-4bfc-a2aa-61cb312d4f93 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:31.273859 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214c634-1051-4bfc-a2aa-61cb312d4f93 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:31.274137 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214c634-1051-4bfc-a2aa-61cb312d4f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:31.310879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-9214c634-1051-4bfc-a2aa-61cb312d4f93 None None] MFA Rules not processed for user `5dd91094a71448b8a6bf44be2c2b1d5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:31.312336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9214c634-1051-4bfc-a2aa-61cb312d4f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:31.393353 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 921/1846] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:31 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:31.398617 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:31.406149 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852da61f-fc1a-47b3-8ebe-8f7362685fac admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:31.408044 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-852da61f-fc1a-47b3-8ebe-8f7362685fac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:31.484667 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852da61f-fc1a-47b3-8ebe-8f7362685fac admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:31.488469 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0350d47c-5518-41df-975e-a17ce46aaa37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:31.489126 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350d47c-5518-41df-975e-a17ce46aaa37 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:31.489440 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350d47c-5518-41df-975e-a17ce46aaa37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:31.489721 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350d47c-5518-41df-975e-a17ce46aaa37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:31.502298 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0350d47c-5518-41df-975e-a17ce46aaa37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:31.504319 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0350d47c-5518-41df-975e-a17ce46aaa37 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:31.515924 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 926/1847] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:31 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:52:31.755289 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214c634-1051-4bfc-a2aa-61cb312d4f93 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:31.765319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214c634-1051-4bfc-a2aa-61cb312d4f93 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:31.769007 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788623e-c509-4510-87a3-3749062f7565 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:31.769628 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788623e-c509-4510-87a3-3749062f7565 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:31.769916 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788623e-c509-4510-87a3-3749062f7565 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:31.770195 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788623e-c509-4510-87a3-3749062f7565 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:31.834051 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a788623e-c509-4510-87a3-3749062f7565 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:31.836145 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a788623e-c509-4510-87a3-3749062f7565 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:31.845468 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 922/1848] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:31 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:52:33.118777 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:34.284732 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:35.101783 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:35.781464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:36.330020 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:36.741445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:37.135920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:37.593752 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:38.199358 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:38.438844 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:38.936943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0350d47c-5518-41df-975e-a17ce46aaa37 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:38.939836 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0350d47c-5518-41df-975e-a17ce46aaa37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:39.027850 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0350d47c-5518-41df-975e-a17ce46aaa37 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:39.033199 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1f519-9cbe-49c3-adab-f658af50e138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:39.033871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1f519-9cbe-49c3-adab-f658af50e138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:39.034155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1f519-9cbe-49c3-adab-f658af50e138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:39.034433 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1f519-9cbe-49c3-adab-f658af50e138 admin admin] PATH_INFO: `/v3/users/5b98f1449be74722ad14ca253365988b` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:39.035814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1f519-9cbe-49c3-adab-f658af50e138 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b98f1449be74722ad14ca253365988b)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:39.038161 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1f519-9cbe-49c3-adab-f658af50e138 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:39.294380 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-39a1f519-9cbe-49c3-adab-f658af50e138 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b98f1449be74722ad14ca253365988b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:39.295072 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-39a1f519-9cbe-49c3-adab-f658af50e138 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b98f1449be74722ad14ca253365988b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:39.308461 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-39a1f519-9cbe-49c3-adab-f658af50e138 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b98f1449be74722ad14ca253365988b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:39.315379 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-39a1f519-9cbe-49c3-adab-f658af50e138 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b98f1449be74722ad14ca253365988b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:39.316146 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-39a1f519-9cbe-49c3-adab-f658af50e138 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b98f1449be74722ad14ca253365988b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:39.428560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 927/1849] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:52:38 2022] DELETE /identity/v3/users/5b98f1449be74722ad14ca253365988b => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:52:39.433263 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da754a-288f-4ee4-b625-d79bb8c64514 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:39.433708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da754a-288f-4ee4-b625-d79bb8c64514 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:39.433831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da754a-288f-4ee4-b625-d79bb8c64514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:39.453734 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a1da754a-288f-4ee4-b625-d79bb8c64514 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:39.455064 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a1da754a-288f-4ee4-b625-d79bb8c64514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:39.515693 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 923/1850] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:39.525595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1f519-9cbe-49c3-adab-f658af50e138 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:39.527094 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-39a1f519-9cbe-49c3-adab-f658af50e138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:39.586087 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1f519-9cbe-49c3-adab-f658af50e138 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:39.590942 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388a402-0666-4123-a948-6c26bd464717 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:39.591836 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388a402-0666-4123-a948-6c26bd464717 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:39.592162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388a402-0666-4123-a948-6c26bd464717 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:39.592545 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388a402-0666-4123-a948-6c26bd464717 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:39.600006 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:39.603987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388a402-0666-4123-a948-6c26bd464717 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:39.605621 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388a402-0666-4123-a948-6c26bd464717 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:39.616088 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 928/1851] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:52:39.868146 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:40.278339 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da754a-288f-4ee4-b625-d79bb8c64514 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:40.280069 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a1da754a-288f-4ee4-b625-d79bb8c64514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:40.300658 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da754a-288f-4ee4-b625-d79bb8c64514 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:40.307374 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817bb794-1782-4021-b769-4a88d380770b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:40.308444 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817bb794-1782-4021-b769-4a88d380770b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:40.308919 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817bb794-1782-4021-b769-4a88d380770b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:40.309615 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817bb794-1782-4021-b769-4a88d380770b admin admin] PATH_INFO: `/v3/projects/9018edf54c1c4a2197591cc99a814bde` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:40.311745 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817bb794-1782-4021-b769-4a88d380770b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9018edf54c1c4a2197591cc99a814bde)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:40.326234 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817bb794-1782-4021-b769-4a88d380770b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:40.531555 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-817bb794-1782-4021-b769-4a88d380770b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9018edf54c1c4a2197591cc99a814bde', 'request_id': 'req-817bb794-1782-4021-b769-4a88d380770b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:40.537100 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-817bb794-1782-4021-b769-4a88d380770b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9018edf54c1c4a2197591cc99a814bde', 'request_id': 'req-817bb794-1782-4021-b769-4a88d380770b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:40.639512 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-817bb794-1782-4021-b769-4a88d380770b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9018edf54c1c4a2197591cc99a814bde', 'request_id': 'req-817bb794-1782-4021-b769-4a88d380770b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:40.657209 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-817bb794-1782-4021-b769-4a88d380770b admin admin] The token cache is being invalidate because project 9018edf54c1c4a2197591cc99a814bde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:52:40.657209 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-817bb794-1782-4021-b769-4a88d380770b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:40.657209 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-817bb794-1782-4021-b769-4a88d380770b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:40.657209 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 924/1852] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:52:40 2022] DELETE /identity/v3/projects/9018edf54c1c4a2197591cc99a814bde => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:40.659699 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388a402-0666-4123-a948-6c26bd464717 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:40.670246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388a402-0666-4123-a948-6c26bd464717 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:40.727336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f3897-c791-48b9-811d-30977ed126e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:40.728059 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f3897-c791-48b9-811d-30977ed126e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:40.728341 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f3897-c791-48b9-811d-30977ed126e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:40.728620 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f3897-c791-48b9-811d-30977ed126e0 admin admin] PATH_INFO: `/v3/users/4edf96e86ce249b283265424c7880c40` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:40.729783 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6f3897-c791-48b9-811d-30977ed126e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4edf96e86ce249b283265424c7880c40)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:40.731753 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6f3897-c791-48b9-811d-30977ed126e0 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:40.822324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:40.979723 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0b6f3897-c791-48b9-811d-30977ed126e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4edf96e86ce249b283265424c7880c40'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:40.980755 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0b6f3897-c791-48b9-811d-30977ed126e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4edf96e86ce249b283265424c7880c40'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:40.994652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0b6f3897-c791-48b9-811d-30977ed126e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4edf96e86ce249b283265424c7880c40'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:41.001988 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0b6f3897-c791-48b9-811d-30977ed126e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4edf96e86ce249b283265424c7880c40'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:41.002883 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0b6f3897-c791-48b9-811d-30977ed126e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4edf96e86ce249b283265424c7880c40'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:41.096000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 929/1853] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:52:40 2022] DELETE /identity/v3/users/4edf96e86ce249b283265424c7880c40 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:41.101811 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:41.645776 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817bb794-1782-4021-b769-4a88d380770b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:41.647260 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-817bb794-1782-4021-b769-4a88d380770b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:41.661781 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817bb794-1782-4021-b769-4a88d380770b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:41.718702 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c323898-4afe-4417-a1c0-8f6dc860ac9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:41.719347 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c323898-4afe-4417-a1c0-8f6dc860ac9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:41.719662 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c323898-4afe-4417-a1c0-8f6dc860ac9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:41.719937 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c323898-4afe-4417-a1c0-8f6dc860ac9a admin admin] PATH_INFO: `/v3/projects/8534fae5fc9a487cb6b298cd03b2146f` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:41.721094 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c323898-4afe-4417-a1c0-8f6dc860ac9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8534fae5fc9a487cb6b298cd03b2146f)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:41.723005 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c323898-4afe-4417-a1c0-8f6dc860ac9a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:41.919837 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9c323898-4afe-4417-a1c0-8f6dc860ac9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8534fae5fc9a487cb6b298cd03b2146f', 'request_id': 'req-9c323898-4afe-4417-a1c0-8f6dc860ac9a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:41.925278 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9c323898-4afe-4417-a1c0-8f6dc860ac9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8534fae5fc9a487cb6b298cd03b2146f', 'request_id': 'req-9c323898-4afe-4417-a1c0-8f6dc860ac9a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:42.073592 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9c323898-4afe-4417-a1c0-8f6dc860ac9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8534fae5fc9a487cb6b298cd03b2146f', 'request_id': 'req-9c323898-4afe-4417-a1c0-8f6dc860ac9a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:42.086486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9c323898-4afe-4417-a1c0-8f6dc860ac9a admin admin] The token cache is being invalidate because project 8534fae5fc9a487cb6b298cd03b2146f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:52:42.087054 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9c323898-4afe-4417-a1c0-8f6dc860ac9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:42.087757 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9c323898-4afe-4417-a1c0-8f6dc860ac9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:42.093965 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 925/1854] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:52:41 2022] DELETE /identity/v3/projects/8534fae5fc9a487cb6b298cd03b2146f => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:42.096130 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cbde3c-f676-4590-b543-a0727eccbe27 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:42.096624 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cbde3c-f676-4590-b543-a0727eccbe27 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:42.096741 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cbde3c-f676-4590-b543-a0727eccbe27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:42.116868 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-f0cbde3c-f676-4590-b543-a0727eccbe27 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:42.118769 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f0cbde3c-f676-4590-b543-a0727eccbe27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:42.185717 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 930/1855] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:42.193663 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7978f5-cd50-49c3-b9ef-178def24a298 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:42.193952 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7978f5-cd50-49c3-b9ef-178def24a298 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:42.194233 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7978f5-cd50-49c3-b9ef-178def24a298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:42.216047 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-8d7978f5-cd50-49c3-b9ef-178def24a298 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:42.217279 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8d7978f5-cd50-49c3-b9ef-178def24a298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:42.225174 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 926/1856] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:42.230034 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cbde3c-f676-4590-b543-a0727eccbe27 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:42.239705 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cbde3c-f676-4590-b543-a0727eccbe27 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:42.244400 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837633e-e463-46bb-a172-82dee6d4d40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:42.245014 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837633e-e463-46bb-a172-82dee6d4d40d admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:42.245402 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837633e-e463-46bb-a172-82dee6d4d40d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:42.249250 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837633e-e463-46bb-a172-82dee6d4d40d admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:42.249250 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837633e-e463-46bb-a172-82dee6d4d40d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:52:42.249250 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837633e-e463-46bb-a172-82dee6d4d40d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:42.249250 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837633e-e463-46bb-a172-82dee6d4d40d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:42.260443 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 931/1857] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:52:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:52:42.264865 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7978f5-cd50-49c3-b9ef-178def24a298 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:42.274749 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7978f5-cd50-49c3-b9ef-178def24a298 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:42.279606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305d76e-7319-4d03-b354-20ca716ee686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:42.280217 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305d76e-7319-4d03-b354-20ca716ee686 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:42.280590 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305d76e-7319-4d03-b354-20ca716ee686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:42.280909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305d76e-7319-4d03-b354-20ca716ee686 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:42.281577 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b305d76e-7319-4d03-b354-20ca716ee686 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:42.283599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b305d76e-7319-4d03-b354-20ca716ee686 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:42.388110 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:42.423287 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 927/1858] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:52:42 2022] POST /identity/v3/projects => generated 389 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:52:42.428079 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837633e-e463-46bb-a172-82dee6d4d40d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:42.438377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837633e-e463-46bb-a172-82dee6d4d40d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:42.496033 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e1773-6ee2-455d-b323-066cccd8e3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:42.496667 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e1773-6ee2-455d-b323-066cccd8e3b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:42.496954 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e1773-6ee2-455d-b323-066cccd8e3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:42.497231 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e1773-6ee2-455d-b323-066cccd8e3b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:42.497913 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e1773-6ee2-455d-b323-066cccd8e3b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:42.499874 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e1773-6ee2-455d-b323-066cccd8e3b8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:42.542493 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:42.669685 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 932/1859] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:52:42 2022] POST /identity/v3/users => generated 295 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:52:42.673755 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305d76e-7319-4d03-b354-20ca716ee686 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:42.684830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305d76e-7319-4d03-b354-20ca716ee686 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:42.689993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87206549-8ef8-4019-b467-715b2f61ce16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:42.690626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87206549-8ef8-4019-b467-715b2f61ce16 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:42.690911 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87206549-8ef8-4019-b467-715b2f61ce16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:42.691194 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87206549-8ef8-4019-b467-715b2f61ce16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:42.691966 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87206549-8ef8-4019-b467-715b2f61ce16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:42.694938 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87206549-8ef8-4019-b467-715b2f61ce16 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:42.705599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 928/1860] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:52:42.722629 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e1773-6ee2-455d-b323-066cccd8e3b8 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:42.743390 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e1773-6ee2-455d-b323-066cccd8e3b8 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:42.748717 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee300c-adba-4d17-add7-9906528dcc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:42.749422 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee300c-adba-4d17-add7-9906528dcc53 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:42.749828 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee300c-adba-4d17-add7-9906528dcc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:42.750180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee300c-adba-4d17-add7-9906528dcc53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:42.751142 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee300c-adba-4d17-add7-9906528dcc53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:42.753344 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee300c-adba-4d17-add7-9906528dcc53 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:42.770850 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 933/1861] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:52:42.774501 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87206549-8ef8-4019-b467-715b2f61ce16 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:42.785187 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87206549-8ef8-4019-b467-715b2f61ce16 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:42.790817 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc1a806-e3f3-4062-bd75-e910a68153ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:42.791540 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1a806-e3f3-4062-bd75-e910a68153ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:42.792034 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1a806-e3f3-4062-bd75-e910a68153ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:42.792313 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1a806-e3f3-4062-bd75-e910a68153ff admin admin] PATH_INFO: `/v3/projects/d9156b266b3943c5ae89902e9b8ba4c2/users/9ec528421da54efea4aa03d086bd191a/roles/e033961a9363456a8510c34414dd839e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:42.807510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc1a806-e3f3-4062-bd75-e910a68153ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9156b266b3943c5ae89902e9b8ba4c2, user_id=9ec528421da54efea4aa03d086bd191a, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:42.810318 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc1a806-e3f3-4062-bd75-e910a68153ff admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:42.925274 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 929/1862] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:52:42 2022] PUT /identity/v3/projects/d9156b266b3943c5ae89902e9b8ba4c2/users/9ec528421da54efea4aa03d086bd191a/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:42.937661 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20878688-da8d-4aeb-a051-3d71a16f2893 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:42.937866 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20878688-da8d-4aeb-a051-3d71a16f2893 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:42.938147 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20878688-da8d-4aeb-a051-3d71a16f2893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:42.967376 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-20878688-da8d-4aeb-a051-3d71a16f2893 None None] MFA Rules not processed for user `9ec528421da54efea4aa03d086bd191a`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:42.968639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-20878688-da8d-4aeb-a051-3d71a16f2893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:43.046732 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 934/1863] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:52:43.051330 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc1a806-e3f3-4062-bd75-e910a68153ff admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:43.120621 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc1a806-e3f3-4062-bd75-e910a68153ff admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:43.125433 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88514e7f-2f06-435a-a15f-37efdf5a8bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:43.126169 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88514e7f-2f06-435a-a15f-37efdf5a8bee admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:43.126516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88514e7f-2f06-435a-a15f-37efdf5a8bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:43.127797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88514e7f-2f06-435a-a15f-37efdf5a8bee admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:43.127992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88514e7f-2f06-435a-a15f-37efdf5a8bee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:43.130039 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88514e7f-2f06-435a-a15f-37efdf5a8bee admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:43.180485 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52309e53-0b76-47b4-a1d2-e55a8e370493 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:43.181073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52309e53-0b76-47b4-a1d2-e55a8e370493 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:43.181073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52309e53-0b76-47b4-a1d2-e55a8e370493 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:43.182413 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 935/1864] 10.222.0.97 () {54 vars in 921 bytes} [Tue Jul 26 14:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:52:43.197743 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f579033-d4b5-49fc-a0eb-46f087e145b5 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:43.198067 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f579033-d4b5-49fc-a0eb-46f087e145b5 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:43.198276 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f579033-d4b5-49fc-a0eb-46f087e145b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:43.220816 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-7f579033-d4b5-49fc-a0eb-46f087e145b5 None None] MFA Rules not processed for user `266abef6cda5495b897170f92043aa03`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:43.222234 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7f579033-d4b5-49fc-a0eb-46f087e145b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:43.245821 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 930/1865] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:52:43 2022] POST /identity/v3/projects => generated 387 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:52:43.251011 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88514e7f-2f06-435a-a15f-37efdf5a8bee admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:43.320202 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 936/1866] 10.222.0.97 () {58 vars in 1018 bytes} [Tue Jul 26 14:52:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:52:43.333992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88514e7f-2f06-435a-a15f-37efdf5a8bee admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:43.339875 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4211a9c-5726-4245-b48d-8d3939156858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:43.340732 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4211a9c-5726-4245-b48d-8d3939156858 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:43.341100 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4211a9c-5726-4245-b48d-8d3939156858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:43.341442 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4211a9c-5726-4245-b48d-8d3939156858 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:43.343722 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4211a9c-5726-4245-b48d-8d3939156858 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:43.345894 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4211a9c-5726-4245-b48d-8d3939156858 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:43.479120 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f579033-d4b5-49fc-a0eb-46f087e145b5 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:43.481097 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7f579033-d4b5-49fc-a0eb-46f087e145b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:43.503060 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f579033-d4b5-49fc-a0eb-46f087e145b5 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:43.515551 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 931/1867] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:52:43 2022] POST /identity/v3/users => generated 300 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:52:43.521578 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4211a9c-5726-4245-b48d-8d3939156858 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:43.534619 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4211a9c-5726-4245-b48d-8d3939156858 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:43.542773 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8936ed6d-d374-4ee0-aaeb-2c2d7956cad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:43.544839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8936ed6d-d374-4ee0-aaeb-2c2d7956cad9 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:43.544839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8936ed6d-d374-4ee0-aaeb-2c2d7956cad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:43.544839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8936ed6d-d374-4ee0-aaeb-2c2d7956cad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:43.545205 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8936ed6d-d374-4ee0-aaeb-2c2d7956cad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:43.547355 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8936ed6d-d374-4ee0-aaeb-2c2d7956cad9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:43.563072 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 932/1868] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:52:43.567226 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8936ed6d-d374-4ee0-aaeb-2c2d7956cad9 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:43.581825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b423d1-5db9-4d1b-a77a-e953ed5bc352 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:43.582601 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b423d1-5db9-4d1b-a77a-e953ed5bc352 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:43.582957 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b423d1-5db9-4d1b-a77a-e953ed5bc352 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:43.583395 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b423d1-5db9-4d1b-a77a-e953ed5bc352 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:43.584117 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8936ed6d-d374-4ee0-aaeb-2c2d7956cad9 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:43.595585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e315d0d-bee9-4879-b255-bed57a6b24fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:43.595585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e315d0d-bee9-4879-b255-bed57a6b24fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:43.595585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e315d0d-bee9-4879-b255-bed57a6b24fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:43.595585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e315d0d-bee9-4879-b255-bed57a6b24fe admin admin] PATH_INFO: `/v3/projects/e452229867e748c3bf7aef356bc1165b/users/d16570e2fcd14f18930833ade365b3f9/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:43.595887 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b423d1-5db9-4d1b-a77a-e953ed5bc352 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:43.597141 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b423d1-5db9-4d1b-a77a-e953ed5bc352 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:43.612615 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 937/1869] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:52:43.619279 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e315d0d-bee9-4879-b255-bed57a6b24fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=e452229867e748c3bf7aef356bc1165b, user_id=d16570e2fcd14f18930833ade365b3f9, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:43.622385 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e315d0d-bee9-4879-b255-bed57a6b24fe admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:43.698434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:43.725657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 933/1870] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:52:43 2022] PUT /identity/v3/projects/e452229867e748c3bf7aef356bc1165b/users/d16570e2fcd14f18930833ade365b3f9/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:43.735530 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b423d1-5db9-4d1b-a77a-e953ed5bc352 service nova] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:43.767891 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b423d1-5db9-4d1b-a77a-e953ed5bc352 service nova] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:43.864430 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662c59e-1c7e-4c41-81d8-db4ca7764d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:43.865350 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662c59e-1c7e-4c41-81d8-db4ca7764d03 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:43.865637 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662c59e-1c7e-4c41-81d8-db4ca7764d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:43.865914 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662c59e-1c7e-4c41-81d8-db4ca7764d03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:43.866740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c662c59e-1c7e-4c41-81d8-db4ca7764d03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:43.868573 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c662c59e-1c7e-4c41-81d8-db4ca7764d03 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:43.887542 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 938/1871] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:52:43.891358 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e315d0d-bee9-4879-b255-bed57a6b24fe admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:43.902290 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e315d0d-bee9-4879-b255-bed57a6b24fe admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:43.907769 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a338d3-4308-48b7-b7d2-f6401b09f225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:43.908456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a338d3-4308-48b7-b7d2-f6401b09f225 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:43.908740 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a338d3-4308-48b7-b7d2-f6401b09f225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:43.909018 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a338d3-4308-48b7-b7d2-f6401b09f225 admin admin] PATH_INFO: `/v3/projects/e452229867e748c3bf7aef356bc1165b/users/d16570e2fcd14f18930833ade365b3f9/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:43.911218 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a338d3-4308-48b7-b7d2-f6401b09f225 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e452229867e748c3bf7aef356bc1165b, user_id=d16570e2fcd14f18930833ade365b3f9, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:43.915260 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a338d3-4308-48b7-b7d2-f6401b09f225 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:43.936868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 934/1872] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:52:43 2022] PUT /identity/v3/projects/e452229867e748c3bf7aef356bc1165b/users/d16570e2fcd14f18930833ade365b3f9/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:52:43.952817 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7933d83-8a03-4efb-9514-25952d074889 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:43.953244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7933d83-8a03-4efb-9514-25952d074889 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:43.953802 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7933d83-8a03-4efb-9514-25952d074889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:44.004762 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e7933d83-8a03-4efb-9514-25952d074889 None None] MFA Rules not processed for user `d16570e2fcd14f18930833ade365b3f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:44.006348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e7933d83-8a03-4efb-9514-25952d074889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:44.120427 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 939/1873] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:43 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:44.291697 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:44.298182 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a338d3-4308-48b7-b7d2-f6401b09f225 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:44.299918 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-03a338d3-4308-48b7-b7d2-f6401b09f225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:44.357293 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a338d3-4308-48b7-b7d2-f6401b09f225 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:44.362877 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678eb8ac-bc90-4a2c-b423-9c17c2fdaf1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:44.362877 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678eb8ac-bc90-4a2c-b423-9c17c2fdaf1d service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:44.362877 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678eb8ac-bc90-4a2c-b423-9c17c2fdaf1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:44.362877 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678eb8ac-bc90-4a2c-b423-9c17c2fdaf1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:44.371381 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678eb8ac-bc90-4a2c-b423-9c17c2fdaf1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:44.372814 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678eb8ac-bc90-4a2c-b423-9c17c2fdaf1d service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:44.383027 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 935/1874] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:44 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:52:44.656968 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7933d83-8a03-4efb-9514-25952d074889 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:44.658592 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e7933d83-8a03-4efb-9514-25952d074889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:44.725265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7933d83-8a03-4efb-9514-25952d074889 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:44.725409 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678eb8ac-bc90-4a2c-b423-9c17c2fdaf1d service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:44.730321 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c93096-e7a8-4609-91ae-36b09d92ab0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:44.731009 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c93096-e7a8-4609-91ae-36b09d92ab0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:44.731292 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c93096-e7a8-4609-91ae-36b09d92ab0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:44.731694 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c93096-e7a8-4609-91ae-36b09d92ab0e admin admin] PATH_INFO: `/v3/users/6890768d36fe4f34b661569372025201` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:44.732991 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c93096-e7a8-4609-91ae-36b09d92ab0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6890768d36fe4f34b661569372025201)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:44.735135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c93096-e7a8-4609-91ae-36b09d92ab0e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:44.738877 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678eb8ac-bc90-4a2c-b423-9c17c2fdaf1d service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:44.747163 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609967e8-53fb-4d7d-a9de-33e3e0a913f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:44.748060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609967e8-53fb-4d7d-a9de-33e3e0a913f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:44.748480 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609967e8-53fb-4d7d-a9de-33e3e0a913f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:44.748950 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609967e8-53fb-4d7d-a9de-33e3e0a913f7 admin admin] PATH_INFO: `/v3/users/9ec528421da54efea4aa03d086bd191a` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:44.750786 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609967e8-53fb-4d7d-a9de-33e3e0a913f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ec528421da54efea4aa03d086bd191a)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:44.754310 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609967e8-53fb-4d7d-a9de-33e3e0a913f7 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:45.001977 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e5c93096-e7a8-4609-91ae-36b09d92ab0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6890768d36fe4f34b661569372025201'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:45.002977 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e5c93096-e7a8-4609-91ae-36b09d92ab0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6890768d36fe4f34b661569372025201'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:45.003215 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-609967e8-53fb-4d7d-a9de-33e3e0a913f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ec528421da54efea4aa03d086bd191a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:45.004050 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-609967e8-53fb-4d7d-a9de-33e3e0a913f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ec528421da54efea4aa03d086bd191a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:45.019935 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-609967e8-53fb-4d7d-a9de-33e3e0a913f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ec528421da54efea4aa03d086bd191a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:45.029234 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-609967e8-53fb-4d7d-a9de-33e3e0a913f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ec528421da54efea4aa03d086bd191a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:45.030059 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-609967e8-53fb-4d7d-a9de-33e3e0a913f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ec528421da54efea4aa03d086bd191a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:45.033484 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e5c93096-e7a8-4609-91ae-36b09d92ab0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6890768d36fe4f34b661569372025201'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:45.076497 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e5c93096-e7a8-4609-91ae-36b09d92ab0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6890768d36fe4f34b661569372025201'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:45.078489 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e5c93096-e7a8-4609-91ae-36b09d92ab0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6890768d36fe4f34b661569372025201'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:45.148414 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 936/1875] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:52:44 2022] DELETE /identity/v3/users/9ec528421da54efea4aa03d086bd191a => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:52:45.155719 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833fc004-3526-463b-b96b-ab41705cabf2 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:45.155719 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833fc004-3526-463b-b96b-ab41705cabf2 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:45.155719 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833fc004-3526-463b-b96b-ab41705cabf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:45.189706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-833fc004-3526-463b-b96b-ab41705cabf2 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:45.191154 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-833fc004-3526-463b-b96b-ab41705cabf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:45.259904 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 940/1876] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:52:44 2022] DELETE /identity/v3/users/6890768d36fe4f34b661569372025201 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:45.272558 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea663b46-dcd4-4cc7-ac90-769eb2fca862 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:45.273249 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea663b46-dcd4-4cc7-ac90-769eb2fca862 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:45.274701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea663b46-dcd4-4cc7-ac90-769eb2fca862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:45.294149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 937/1877] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:45.305492 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833fc004-3526-463b-b96b-ab41705cabf2 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:45.306043 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-833fc004-3526-463b-b96b-ab41705cabf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:45.306685 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ea663b46-dcd4-4cc7-ac90-769eb2fca862 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:45.308028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ea663b46-dcd4-4cc7-ac90-769eb2fca862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:45.370852 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833fc004-3526-463b-b96b-ab41705cabf2 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:45.378605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 941/1878] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:52:45.389216 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea663b46-dcd4-4cc7-ac90-769eb2fca862 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:45.428551 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1f816-2921-4bde-b8a1-231c8ee0d436 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:45.429314 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1f816-2921-4bde-b8a1-231c8ee0d436 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:45.429701 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1f816-2921-4bde-b8a1-231c8ee0d436 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:45.430274 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1f816-2921-4bde-b8a1-231c8ee0d436 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:45.440717 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b1f816-2921-4bde-b8a1-231c8ee0d436 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:45.442310 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b1f816-2921-4bde-b8a1-231c8ee0d436 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:45.453779 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 938/1879] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:52:45.515451 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea663b46-dcd4-4cc7-ac90-769eb2fca862 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:45.520683 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f45a3-81e2-4e66-9dcc-c110454c675c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:45.521577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f45a3-81e2-4e66-9dcc-c110454c675c service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:45.521723 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f45a3-81e2-4e66-9dcc-c110454c675c service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:45.522035 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f45a3-81e2-4e66-9dcc-c110454c675c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:45.543924 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f45a3-81e2-4e66-9dcc-c110454c675c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:45.546232 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f45a3-81e2-4e66-9dcc-c110454c675c service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:45.559509 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 942/1880] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:52:45.999941 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:46.127009 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1f816-2921-4bde-b8a1-231c8ee0d436 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:46.127748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f45a3-81e2-4e66-9dcc-c110454c675c service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:46.129343 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4e7f45a3-81e2-4e66-9dcc-c110454c675c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:46.129802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-84b1f816-2921-4bde-b8a1-231c8ee0d436 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:46.144434 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1f816-2921-4bde-b8a1-231c8ee0d436 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:46.149808 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02f663-1ea8-436c-b82f-002baf58908c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:46.150469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02f663-1ea8-436c-b82f-002baf58908c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:46.150779 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02f663-1ea8-436c-b82f-002baf58908c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:46.151029 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02f663-1ea8-436c-b82f-002baf58908c admin admin] PATH_INFO: `/v3/projects/8d066e1eae3e4e9c84261b9bfb988120` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:46.152340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf02f663-1ea8-436c-b82f-002baf58908c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d066e1eae3e4e9c84261b9bfb988120)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:46.154815 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf02f663-1ea8-436c-b82f-002baf58908c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:46.160670 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f45a3-81e2-4e66-9dcc-c110454c675c service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:46.166977 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe77aca-faa1-4898-a999-ca5cd7886e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:46.167669 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe77aca-faa1-4898-a999-ca5cd7886e09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:46.167957 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe77aca-faa1-4898-a999-ca5cd7886e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:46.168295 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe77aca-faa1-4898-a999-ca5cd7886e09 admin admin] PATH_INFO: `/v3/projects/d9156b266b3943c5ae89902e9b8ba4c2` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:46.169547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe77aca-faa1-4898-a999-ca5cd7886e09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9156b266b3943c5ae89902e9b8ba4c2)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:46.171634 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe77aca-faa1-4898-a999-ca5cd7886e09 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:46.272441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-bf02f663-1ea8-436c-b82f-002baf58908c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d066e1eae3e4e9c84261b9bfb988120', 'request_id': 'req-bf02f663-1ea8-436c-b82f-002baf58908c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:46.280694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-bf02f663-1ea8-436c-b82f-002baf58908c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d066e1eae3e4e9c84261b9bfb988120', 'request_id': 'req-bf02f663-1ea8-436c-b82f-002baf58908c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:46.305320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fbe77aca-faa1-4898-a999-ca5cd7886e09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9156b266b3943c5ae89902e9b8ba4c2', 'request_id': 'req-fbe77aca-faa1-4898-a999-ca5cd7886e09'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:46.313930 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fbe77aca-faa1-4898-a999-ca5cd7886e09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9156b266b3943c5ae89902e9b8ba4c2', 'request_id': 'req-fbe77aca-faa1-4898-a999-ca5cd7886e09'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:46.370959 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-bf02f663-1ea8-436c-b82f-002baf58908c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d066e1eae3e4e9c84261b9bfb988120', 'request_id': 'req-bf02f663-1ea8-436c-b82f-002baf58908c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:46.385356 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-bf02f663-1ea8-436c-b82f-002baf58908c admin admin] The token cache is being invalidate because project 8d066e1eae3e4e9c84261b9bfb988120 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:52:46.386141 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-bf02f663-1ea8-436c-b82f-002baf58908c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:46.386897 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-bf02f663-1ea8-436c-b82f-002baf58908c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:46.388900 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 939/1881] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:52:46 2022] DELETE /identity/v3/projects/8d066e1eae3e4e9c84261b9bfb988120 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:46.392573 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02f663-1ea8-436c-b82f-002baf58908c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:46.394090 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-bf02f663-1ea8-436c-b82f-002baf58908c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:46.463662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fbe77aca-faa1-4898-a999-ca5cd7886e09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9156b266b3943c5ae89902e9b8ba4c2', 'request_id': 'req-fbe77aca-faa1-4898-a999-ca5cd7886e09'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:46.468339 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02f663-1ea8-436c-b82f-002baf58908c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:46.481740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fbe77aca-faa1-4898-a999-ca5cd7886e09 admin admin] The token cache is being invalidate because project d9156b266b3943c5ae89902e9b8ba4c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:52:46.482039 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fbe77aca-faa1-4898-a999-ca5cd7886e09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:46.484532 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fbe77aca-faa1-4898-a999-ca5cd7886e09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:46.486260 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 943/1882] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:52:46 2022] DELETE /identity/v3/projects/d9156b266b3943c5ae89902e9b8ba4c2 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:46.492167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe77aca-faa1-4898-a999-ca5cd7886e09 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:46.494438 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-fbe77aca-faa1-4898-a999-ca5cd7886e09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:46.539442 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe0949-09bf-4473-8df0-2841cce1c80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:46.540571 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe0949-09bf-4473-8df0-2841cce1c80c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:46.541095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe0949-09bf-4473-8df0-2841cce1c80c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:46.541428 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe0949-09bf-4473-8df0-2841cce1c80c admin admin] PATH_INFO: `/v3/users/536710bbb8014a708c6d88fc25d28cd8` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:46.545994 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe0949-09bf-4473-8df0-2841cce1c80c admin admin] RBAC: Authorizing `identity:delete_user(user_id=536710bbb8014a708c6d88fc25d28cd8)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:46.545994 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe0949-09bf-4473-8df0-2841cce1c80c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:46.582177 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe77aca-faa1-4898-a999-ca5cd7886e09 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:46.587700 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5cd66-c164-4427-aaf4-a5c3f6cf0579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:46.591960 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5cd66-c164-4427-aaf4-a5c3f6cf0579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:46.591960 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5cd66-c164-4427-aaf4-a5c3f6cf0579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:46.591960 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5cd66-c164-4427-aaf4-a5c3f6cf0579 admin admin] PATH_INFO: `/v3/users/d16570e2fcd14f18930833ade365b3f9` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:46.591960 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee5cd66-c164-4427-aaf4-a5c3f6cf0579 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d16570e2fcd14f18930833ade365b3f9)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:46.592486 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee5cd66-c164-4427-aaf4-a5c3f6cf0579 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:47.080962 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cee5cd66-c164-4427-aaf4-a5c3f6cf0579 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd16570e2fcd14f18930833ade365b3f9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:47.082124 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cee5cd66-c164-4427-aaf4-a5c3f6cf0579 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd16570e2fcd14f18930833ade365b3f9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:47.083598 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b8fe0949-09bf-4473-8df0-2841cce1c80c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '536710bbb8014a708c6d88fc25d28cd8'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:47.084540 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b8fe0949-09bf-4473-8df0-2841cce1c80c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '536710bbb8014a708c6d88fc25d28cd8'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:47.108723 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b8fe0949-09bf-4473-8df0-2841cce1c80c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '536710bbb8014a708c6d88fc25d28cd8'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:47.109285 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cee5cd66-c164-4427-aaf4-a5c3f6cf0579 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd16570e2fcd14f18930833ade365b3f9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:47.109365 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b8fe0949-09bf-4473-8df0-2841cce1c80c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '536710bbb8014a708c6d88fc25d28cd8'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:47.109882 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b8fe0949-09bf-4473-8df0-2841cce1c80c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '536710bbb8014a708c6d88fc25d28cd8'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:47.131167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cee5cd66-c164-4427-aaf4-a5c3f6cf0579 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd16570e2fcd14f18930833ade365b3f9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:47.132311 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cee5cd66-c164-4427-aaf4-a5c3f6cf0579 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd16570e2fcd14f18930833ade365b3f9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:47.211051 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 940/1883] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:52:46 2022] DELETE /identity/v3/users/536710bbb8014a708c6d88fc25d28cd8 => generated 0 bytes in 819 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:47.245902 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:47.315381 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 944/1884] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:52:46 2022] DELETE /identity/v3/users/d16570e2fcd14f18930833ade365b3f9 => generated 0 bytes in 825 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:52:47.968283 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe0949-09bf-4473-8df0-2841cce1c80c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:47.973849 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5cd66-c164-4427-aaf4-a5c3f6cf0579 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:47.973849 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-cee5cd66-c164-4427-aaf4-a5c3f6cf0579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:47.974227 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b8fe0949-09bf-4473-8df0-2841cce1c80c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:48.103976 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe0949-09bf-4473-8df0-2841cce1c80c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:48.171010 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5cd66-c164-4427-aaf4-a5c3f6cf0579 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:48.189597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3578c8e-a01c-458f-b95f-386dab3fff7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:48.190434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3578c8e-a01c-458f-b95f-386dab3fff7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:48.190770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3578c8e-a01c-458f-b95f-386dab3fff7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:48.191057 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3578c8e-a01c-458f-b95f-386dab3fff7e admin admin] PATH_INFO: `/v3/projects/4180d8b6c8b5493f9843effef37373a3` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:48.196183 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3578c8e-a01c-458f-b95f-386dab3fff7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4180d8b6c8b5493f9843effef37373a3)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:48.198226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3578c8e-a01c-458f-b95f-386dab3fff7e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:48.207911 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32704b3a-db7b-4774-8a21-0f91e595be1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:48.208607 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32704b3a-db7b-4774-8a21-0f91e595be1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:48.209060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32704b3a-db7b-4774-8a21-0f91e595be1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:48.210610 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32704b3a-db7b-4774-8a21-0f91e595be1a admin admin] PATH_INFO: `/v3/projects/e452229867e748c3bf7aef356bc1165b` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:48.214574 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32704b3a-db7b-4774-8a21-0f91e595be1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e452229867e748c3bf7aef356bc1165b)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:48.218772 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32704b3a-db7b-4774-8a21-0f91e595be1a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:48.392706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:48.420930 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c3578c8e-a01c-458f-b95f-386dab3fff7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4180d8b6c8b5493f9843effef37373a3', 'request_id': 'req-c3578c8e-a01c-458f-b95f-386dab3fff7e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:48.421986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-32704b3a-db7b-4774-8a21-0f91e595be1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e452229867e748c3bf7aef356bc1165b', 'request_id': 'req-32704b3a-db7b-4774-8a21-0f91e595be1a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:48.430881 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-32704b3a-db7b-4774-8a21-0f91e595be1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e452229867e748c3bf7aef356bc1165b', 'request_id': 'req-32704b3a-db7b-4774-8a21-0f91e595be1a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:48.444379 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c3578c8e-a01c-458f-b95f-386dab3fff7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4180d8b6c8b5493f9843effef37373a3', 'request_id': 'req-c3578c8e-a01c-458f-b95f-386dab3fff7e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:48.526690 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c3578c8e-a01c-458f-b95f-386dab3fff7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4180d8b6c8b5493f9843effef37373a3', 'request_id': 'req-c3578c8e-a01c-458f-b95f-386dab3fff7e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:48.542081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c3578c8e-a01c-458f-b95f-386dab3fff7e admin admin] The token cache is being invalidate because project 4180d8b6c8b5493f9843effef37373a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:52:48.542539 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c3578c8e-a01c-458f-b95f-386dab3fff7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:48.543226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c3578c8e-a01c-458f-b95f-386dab3fff7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:48.544819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 945/1885] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:52:47 2022] DELETE /identity/v3/projects/4180d8b6c8b5493f9843effef37373a3 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:48.549609 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad07e3d-ca01-485c-ac36-f489a54fdd21 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:48.550170 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad07e3d-ca01-485c-ac36-f489a54fdd21 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:48.550533 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad07e3d-ca01-485c-ac36-f489a54fdd21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:48.579031 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3ad07e3d-ca01-485c-ac36-f489a54fdd21 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:48.583500 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3ad07e3d-ca01-485c-ac36-f489a54fdd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:48.674230 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-32704b3a-db7b-4774-8a21-0f91e595be1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e452229867e748c3bf7aef356bc1165b', 'request_id': 'req-32704b3a-db7b-4774-8a21-0f91e595be1a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:48.688892 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 946/1886] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:48.690754 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7ea5b-11c3-41cf-93a5-f86f606b87cf None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:48.691053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7ea5b-11c3-41cf-93a5-f86f606b87cf None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:48.691607 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7ea5b-11c3-41cf-93a5-f86f606b87cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:48.707964 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-32704b3a-db7b-4774-8a21-0f91e595be1a admin admin] The token cache is being invalidate because project e452229867e748c3bf7aef356bc1165b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:52:48.708516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-32704b3a-db7b-4774-8a21-0f91e595be1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:48.709385 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-32704b3a-db7b-4774-8a21-0f91e595be1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:48.721453 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 941/1887] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:52:47 2022] DELETE /identity/v3/projects/e452229867e748c3bf7aef356bc1165b => generated 0 bytes in 745 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:48.721453 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf3f55-2545-45a6-8d9e-ba6778c6019b None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:48.721453 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf3f55-2545-45a6-8d9e-ba6778c6019b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:48.721453 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf3f55-2545-45a6-8d9e-ba6778c6019b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:48.752996 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6cbf3f55-2545-45a6-8d9e-ba6778c6019b None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:48.754402 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6cbf3f55-2545-45a6-8d9e-ba6778c6019b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:48.759358 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-25b7ea5b-11c3-41cf-93a5-f86f606b87cf None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:48.764458 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 942/1888] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:48.771044 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-25b7ea5b-11c3-41cf-93a5-f86f606b87cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:48.776062 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f7cd34-0afb-46ec-bfd4-520dc6f21cab None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:48.776911 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f7cd34-0afb-46ec-bfd4-520dc6f21cab None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:48.777985 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f7cd34-0afb-46ec-bfd4-520dc6f21cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:48.784241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 947/1889] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:52:48.786119 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7ea5b-11c3-41cf-93a5-f86f606b87cf None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:48.799791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7ea5b-11c3-41cf-93a5-f86f606b87cf None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:48.806211 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d175e4f9-75f1-4296-87a6-8cba0bd62ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:48.806934 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175e4f9-75f1-4296-87a6-8cba0bd62ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:48.807278 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175e4f9-75f1-4296-87a6-8cba0bd62ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:48.807663 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175e4f9-75f1-4296-87a6-8cba0bd62ba6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:48.811447 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d175e4f9-75f1-4296-87a6-8cba0bd62ba6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:52:48.811447 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d175e4f9-75f1-4296-87a6-8cba0bd62ba6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:48.811447 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d175e4f9-75f1-4296-87a6-8cba0bd62ba6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:48.828970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 948/1890] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:52:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:52:48.829722 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d175e4f9-75f1-4296-87a6-8cba0bd62ba6 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:48.836054 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-51f7cd34-0afb-46ec-bfd4-520dc6f21cab None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:48.841004 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-51f7cd34-0afb-46ec-bfd4-520dc6f21cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:48.843383 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d175e4f9-75f1-4296-87a6-8cba0bd62ba6 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:48.850700 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee79d2-612a-463b-b443-2b01cb3ff3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:48.851417 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee79d2-612a-463b-b443-2b01cb3ff3fc admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:48.851942 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee79d2-612a-463b-b443-2b01cb3ff3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:48.852240 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee79d2-612a-463b-b443-2b01cb3ff3fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:48.853107 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee79d2-612a-463b-b443-2b01cb3ff3fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:48.855541 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee79d2-612a-463b-b443-2b01cb3ff3fc admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:48.859103 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 943/1891] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:48.860203 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f7cd34-0afb-46ec-bfd4-520dc6f21cab None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:48.875052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f7cd34-0afb-46ec-bfd4-520dc6f21cab None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:48.885948 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670b9c1-8b9f-4c94-a56d-72202260634b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:48.886950 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670b9c1-8b9f-4c94-a56d-72202260634b admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:48.887730 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670b9c1-8b9f-4c94-a56d-72202260634b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:48.888126 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670b9c1-8b9f-4c94-a56d-72202260634b admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:48.889389 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e670b9c1-8b9f-4c94-a56d-72202260634b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:52:48.890016 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e670b9c1-8b9f-4c94-a56d-72202260634b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:48.891815 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e670b9c1-8b9f-4c94-a56d-72202260634b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:48.908985 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 944/1892] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:52:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:52:48.914291 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670b9c1-8b9f-4c94-a56d-72202260634b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:48.925888 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670b9c1-8b9f-4c94-a56d-72202260634b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:48.933106 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1bc6e6-ad58-4060-ac87-81412982b967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:48.934023 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bc6e6-ad58-4060-ac87-81412982b967 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:48.934365 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bc6e6-ad58-4060-ac87-81412982b967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:48.934830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bc6e6-ad58-4060-ac87-81412982b967 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:48.935772 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1bc6e6-ad58-4060-ac87-81412982b967 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:48.937937 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1bc6e6-ad58-4060-ac87-81412982b967 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:48.998120 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 949/1893] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:52:48 2022] POST /identity/v3/projects => generated 371 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:52:49.003057 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee79d2-612a-463b-b443-2b01cb3ff3fc admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:49.083245 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee79d2-612a-463b-b443-2b01cb3ff3fc admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:49.088856 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171283a-c575-4b79-8e37-c12f8b3b7a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:49.089654 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171283a-c575-4b79-8e37-c12f8b3b7a31 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:49.089945 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171283a-c575-4b79-8e37-c12f8b3b7a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:49.090227 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171283a-c575-4b79-8e37-c12f8b3b7a31 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:49.090904 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c171283a-c575-4b79-8e37-c12f8b3b7a31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:49.092967 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c171283a-c575-4b79-8e37-c12f8b3b7a31 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:49.107433 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 945/1894] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:52:48 2022] POST /identity/v3/projects => generated 369 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:52:49.111116 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1bc6e6-ad58-4060-ac87-81412982b967 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:49.116782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:49.184515 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1bc6e6-ad58-4060-ac87-81412982b967 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:49.189236 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4479b-a368-48a4-88bd-b3245639a5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:49.195299 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4479b-a368-48a4-88bd-b3245639a5d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:49.195299 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4479b-a368-48a4-88bd-b3245639a5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:49.195299 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4479b-a368-48a4-88bd-b3245639a5d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:49.195299 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4479b-a368-48a4-88bd-b3245639a5d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:49.195299 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4479b-a368-48a4-88bd-b3245639a5d5 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:49.246107 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 950/1895] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:52:49 2022] POST /identity/v3/users => generated 286 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:52:49.250396 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171283a-c575-4b79-8e37-c12f8b3b7a31 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:49.260490 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171283a-c575-4b79-8e37-c12f8b3b7a31 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:49.265189 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebedcde-beeb-49c9-98cb-8e4b66a6f248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:49.265820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebedcde-beeb-49c9-98cb-8e4b66a6f248 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:49.266108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebedcde-beeb-49c9-98cb-8e4b66a6f248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:49.266386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebedcde-beeb-49c9-98cb-8e4b66a6f248 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:49.267345 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebedcde-beeb-49c9-98cb-8e4b66a6f248 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:49.268875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebedcde-beeb-49c9-98cb-8e4b66a6f248 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:49.279341 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 951/1896] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:52:49.284262 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebedcde-beeb-49c9-98cb-8e4b66a6f248 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:49.296881 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebedcde-beeb-49c9-98cb-8e4b66a6f248 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:49.316318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a089836-2229-4000-8275-5dfad3507656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:49.316318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a089836-2229-4000-8275-5dfad3507656 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:49.316318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a089836-2229-4000-8275-5dfad3507656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:49.316318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a089836-2229-4000-8275-5dfad3507656 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:49.317222 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a089836-2229-4000-8275-5dfad3507656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:49.319009 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a089836-2229-4000-8275-5dfad3507656 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:49.340113 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 952/1897] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:52:49.345333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a089836-2229-4000-8275-5dfad3507656 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:49.359029 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 946/1898] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:52:49 2022] POST /identity/v3/users => generated 285 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:52:49.364707 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4479b-a368-48a4-88bd-b3245639a5d5 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:49.374148 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a089836-2229-4000-8275-5dfad3507656 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:49.379720 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf369eeb-aff8-4aff-8ae3-480d7df44d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:49.380542 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf369eeb-aff8-4aff-8ae3-480d7df44d2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:49.380805 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf369eeb-aff8-4aff-8ae3-480d7df44d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:49.381142 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf369eeb-aff8-4aff-8ae3-480d7df44d2c admin admin] PATH_INFO: `/v3/projects/ffe2a45945754ec7a7dac33c7eb5077c/users/222f461ac70d4423baf93df9109c7f08/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:49.382719 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4479b-a368-48a4-88bd-b3245639a5d5 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:49.392207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695f4d3-23bb-448a-8abb-06feead727e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:49.393268 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695f4d3-23bb-448a-8abb-06feead727e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:49.393700 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695f4d3-23bb-448a-8abb-06feead727e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:49.394129 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695f4d3-23bb-448a-8abb-06feead727e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:49.395061 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7695f4d3-23bb-448a-8abb-06feead727e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:49.397265 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7695f4d3-23bb-448a-8abb-06feead727e3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:49.402498 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf369eeb-aff8-4aff-8ae3-480d7df44d2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffe2a45945754ec7a7dac33c7eb5077c, user_id=222f461ac70d4423baf93df9109c7f08, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:49.402498 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf369eeb-aff8-4aff-8ae3-480d7df44d2c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:49.411225 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 947/1899] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:52:49.416174 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695f4d3-23bb-448a-8abb-06feead727e3 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:49.430036 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695f4d3-23bb-448a-8abb-06feead727e3 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:49.435194 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320795a-7a13-4bb9-aae5-4e9c03e074d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:49.435868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320795a-7a13-4bb9-aae5-4e9c03e074d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:49.436156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320795a-7a13-4bb9-aae5-4e9c03e074d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:49.436576 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320795a-7a13-4bb9-aae5-4e9c03e074d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:49.437282 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8320795a-7a13-4bb9-aae5-4e9c03e074d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:49.438934 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8320795a-7a13-4bb9-aae5-4e9c03e074d8 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:49.449902 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 948/1900] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:52:49.454069 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320795a-7a13-4bb9-aae5-4e9c03e074d8 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:49.464253 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320795a-7a13-4bb9-aae5-4e9c03e074d8 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:49.468887 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964abfb-9d27-46b5-a2d7-cf925801239e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:49.469569 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964abfb-9d27-46b5-a2d7-cf925801239e admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:49.469870 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964abfb-9d27-46b5-a2d7-cf925801239e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:49.470151 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964abfb-9d27-46b5-a2d7-cf925801239e admin admin] PATH_INFO: `/v3/projects/378be436866f40589646d60a0eb17c12/users/ba38b8f3289b4a569153410965fc95fe/roles/e033961a9363456a8510c34414dd839e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:49.488298 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f964abfb-9d27-46b5-a2d7-cf925801239e admin admin] RBAC: Authorizing `identity:create_grant(project_id=378be436866f40589646d60a0eb17c12, user_id=ba38b8f3289b4a569153410965fc95fe, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:49.491066 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f964abfb-9d27-46b5-a2d7-cf925801239e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:49.509221 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 953/1901] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:52:49 2022] PUT /identity/v3/projects/ffe2a45945754ec7a7dac33c7eb5077c/users/222f461ac70d4423baf93df9109c7f08/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:52:49.514005 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af219a18-da5c-4317-bed0-b80e5b0e6ab3 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:49.514304 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af219a18-da5c-4317-bed0-b80e5b0e6ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:49.514600 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af219a18-da5c-4317-bed0-b80e5b0e6ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:49.536564 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-af219a18-da5c-4317-bed0-b80e5b0e6ab3 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:49.537891 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-af219a18-da5c-4317-bed0-b80e5b0e6ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:49.616818 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 954/1902] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:52:49.628469 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af219a18-da5c-4317-bed0-b80e5b0e6ab3 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:49.630616 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-af219a18-da5c-4317-bed0-b80e5b0e6ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:49.695604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 949/1903] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:52:49 2022] PUT /identity/v3/projects/378be436866f40589646d60a0eb17c12/users/ba38b8f3289b4a569153410965fc95fe/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:52:49.706445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af219a18-da5c-4317-bed0-b80e5b0e6ab3 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:49.707292 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4d44c-4c99-40dc-8ac3-e161fffd7fdb None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:49.707705 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4d44c-4c99-40dc-8ac3-e161fffd7fdb None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:49.708059 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4d44c-4c99-40dc-8ac3-e161fffd7fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:49.748723 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-9ea4d44c-4c99-40dc-8ac3-e161fffd7fdb None None] MFA Rules not processed for user `ba38b8f3289b4a569153410965fc95fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:49.751345 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9ea4d44c-4c99-40dc-8ac3-e161fffd7fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:49.774228 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc5966-52c3-4e62-9ae0-e72d11205f54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:49.776647 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc5966-52c3-4e62-9ae0-e72d11205f54 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:49.776647 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc5966-52c3-4e62-9ae0-e72d11205f54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:49.776647 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc5966-52c3-4e62-9ae0-e72d11205f54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:49.827201 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 950/1904] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:49 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:49.838477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea4d44c-4c99-40dc-8ac3-e161fffd7fdb None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:49.839904 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9ea4d44c-4c99-40dc-8ac3-e161fffd7fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:49.849595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc5966-52c3-4e62-9ae0-e72d11205f54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:49.851059 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc5966-52c3-4e62-9ae0-e72d11205f54 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:49.862077 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 955/1905] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:52:49.909300 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea4d44c-4c99-40dc-8ac3-e161fffd7fdb None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:49.912891 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1534a5-6050-4997-876f-b0c6e4284700 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:49.913524 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1534a5-6050-4997-876f-b0c6e4284700 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:49.914770 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1534a5-6050-4997-876f-b0c6e4284700 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:49.914770 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1534a5-6050-4997-876f-b0c6e4284700 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:49.924418 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1534a5-6050-4997-876f-b0c6e4284700 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:49.926026 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1534a5-6050-4997-876f-b0c6e4284700 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:49.936960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 951/1906] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:49 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:52:49.960951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc5966-52c3-4e62-9ae0-e72d11205f54 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:49.972895 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc5966-52c3-4e62-9ae0-e72d11205f54 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:49.979031 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f465974c-31f8-4761-944e-cd3f4c388368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:49.979820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465974c-31f8-4761-944e-cd3f4c388368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:49.980175 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465974c-31f8-4761-944e-cd3f4c388368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:49.980576 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465974c-31f8-4761-944e-cd3f4c388368 admin admin] PATH_INFO: `/v3/users/ba38b8f3289b4a569153410965fc95fe` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:49.981861 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f465974c-31f8-4761-944e-cd3f4c388368 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba38b8f3289b4a569153410965fc95fe)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:49.983978 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f465974c-31f8-4761-944e-cd3f4c388368 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:50.204224 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f465974c-31f8-4761-944e-cd3f4c388368 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba38b8f3289b4a569153410965fc95fe'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:50.205041 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f465974c-31f8-4761-944e-cd3f4c388368 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba38b8f3289b4a569153410965fc95fe'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:50.219721 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f465974c-31f8-4761-944e-cd3f4c388368 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba38b8f3289b4a569153410965fc95fe'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:50.228563 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f465974c-31f8-4761-944e-cd3f4c388368 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba38b8f3289b4a569153410965fc95fe'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:50.229307 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f465974c-31f8-4761-944e-cd3f4c388368 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba38b8f3289b4a569153410965fc95fe'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:50.354603 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 956/1907] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:52:49 2022] DELETE /identity/v3/users/ba38b8f3289b4a569153410965fc95fe => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:50.360114 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891270ef-af97-48e4-9c32-1dcefff035e5 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:50.360114 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891270ef-af97-48e4-9c32-1dcefff035e5 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:50.360626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891270ef-af97-48e4-9c32-1dcefff035e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:50.384263 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-891270ef-af97-48e4-9c32-1dcefff035e5 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:50.385732 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-891270ef-af97-48e4-9c32-1dcefff035e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:50.476600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 952/1908] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:50.489777 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f465974c-31f8-4761-944e-cd3f4c388368 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:50.492474 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f465974c-31f8-4761-944e-cd3f4c388368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:50.557427 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f465974c-31f8-4761-944e-cd3f4c388368 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:50.561955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3885bcb-fbbd-4933-97e6-0e38ef3d9ba8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:50.562588 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3885bcb-fbbd-4933-97e6-0e38ef3d9ba8 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:50.562865 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3885bcb-fbbd-4933-97e6-0e38ef3d9ba8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:50.563136 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3885bcb-fbbd-4933-97e6-0e38ef3d9ba8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:50.575209 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3885bcb-fbbd-4933-97e6-0e38ef3d9ba8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:50.577650 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3885bcb-fbbd-4933-97e6-0e38ef3d9ba8 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:50.596805 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 957/1909] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:52:51.085938 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:51.188623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891270ef-af97-48e4-9c32-1dcefff035e5 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:51.190967 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-891270ef-af97-48e4-9c32-1dcefff035e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:51.212333 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891270ef-af97-48e4-9c32-1dcefff035e5 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:51.224653 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda25a34-ca2d-41ab-a4c6-15756f329fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:51.225405 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda25a34-ca2d-41ab-a4c6-15756f329fea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:51.225760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda25a34-ca2d-41ab-a4c6-15756f329fea admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:51.226146 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda25a34-ca2d-41ab-a4c6-15756f329fea admin admin] PATH_INFO: `/v3/projects/378be436866f40589646d60a0eb17c12` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:51.227488 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda25a34-ca2d-41ab-a4c6-15756f329fea admin admin] RBAC: Authorizing `identity:delete_project(project_id=378be436866f40589646d60a0eb17c12)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:51.229595 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda25a34-ca2d-41ab-a4c6-15756f329fea admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:51.234898 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:51.410764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cda25a34-ca2d-41ab-a4c6-15756f329fea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '378be436866f40589646d60a0eb17c12', 'request_id': 'req-cda25a34-ca2d-41ab-a4c6-15756f329fea'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:51.420004 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cda25a34-ca2d-41ab-a4c6-15756f329fea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '378be436866f40589646d60a0eb17c12', 'request_id': 'req-cda25a34-ca2d-41ab-a4c6-15756f329fea'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:51.537568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cda25a34-ca2d-41ab-a4c6-15756f329fea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '378be436866f40589646d60a0eb17c12', 'request_id': 'req-cda25a34-ca2d-41ab-a4c6-15756f329fea'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:51.553131 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cda25a34-ca2d-41ab-a4c6-15756f329fea admin admin] The token cache is being invalidate because project 378be436866f40589646d60a0eb17c12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:52:51.553444 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cda25a34-ca2d-41ab-a4c6-15756f329fea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:51.554140 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cda25a34-ca2d-41ab-a4c6-15756f329fea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:52:51.556754 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 953/1910] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:52:51 2022] DELETE /identity/v3/projects/378be436866f40589646d60a0eb17c12 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:51.561751 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381b963-c39f-410f-a975-80ebc933f3cb None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:51.562245 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381b963-c39f-410f-a975-80ebc933f3cb None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:51.562331 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381b963-c39f-410f-a975-80ebc933f3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:51.586620 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-5381b963-c39f-410f-a975-80ebc933f3cb None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:51.593961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5381b963-c39f-410f-a975-80ebc933f3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:51.668596 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 958/1911] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:51.677492 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca945cde-fa95-48ee-b2e1-888cb57268e5 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:51.677788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca945cde-fa95-48ee-b2e1-888cb57268e5 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:51.678074 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca945cde-fa95-48ee-b2e1-888cb57268e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:51.705803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ca945cde-fa95-48ee-b2e1-888cb57268e5 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:51.707045 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ca945cde-fa95-48ee-b2e1-888cb57268e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:51.716481 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 954/1912] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:51.720983 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5381b963-c39f-410f-a975-80ebc933f3cb None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:51.731475 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5381b963-c39f-410f-a975-80ebc933f3cb None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:51.736244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63971260-2ff1-433d-8eb3-3836226eb37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:51.737059 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63971260-2ff1-433d-8eb3-3836226eb37d admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:51.737344 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63971260-2ff1-433d-8eb3-3836226eb37d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:51.737617 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63971260-2ff1-433d-8eb3-3836226eb37d admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:51.738495 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63971260-2ff1-433d-8eb3-3836226eb37d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:52:51.738803 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63971260-2ff1-433d-8eb3-3836226eb37d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:51.740742 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63971260-2ff1-433d-8eb3-3836226eb37d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:51.757304 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 959/1913] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:52:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:52:51.758650 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca945cde-fa95-48ee-b2e1-888cb57268e5 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:51.769615 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca945cde-fa95-48ee-b2e1-888cb57268e5 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:51.773421 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb631a6-3b27-4391-8e89-03b2c559914a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:51.773421 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb631a6-3b27-4391-8e89-03b2c559914a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:51.773421 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb631a6-3b27-4391-8e89-03b2c559914a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:51.780150 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ef6572-52f7-4d6b-a2e5-2b359a8749bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:51.780794 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef6572-52f7-4d6b-a2e5-2b359a8749bd admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:51.781356 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef6572-52f7-4d6b-a2e5-2b359a8749bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:51.781669 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef6572-52f7-4d6b-a2e5-2b359a8749bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:51.785883 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ef6572-52f7-4d6b-a2e5-2b359a8749bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:51.785883 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ef6572-52f7-4d6b-a2e5-2b359a8749bd admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:51.805445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-cdb631a6-3b27-4391-8e89-03b2c559914a None None] MFA Rules not processed for user `222f461ac70d4423baf93df9109c7f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:51.807241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-cdb631a6-3b27-4391-8e89-03b2c559914a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:51.876164 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 960/1914] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:51.881476 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 955/1915] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:52:51 2022] POST /identity/v3/projects => generated 389 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:52:51.882001 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb631a6-3b27-4391-8e89-03b2c559914a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:51.884911 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-cdb631a6-3b27-4391-8e89-03b2c559914a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:51.885804 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ef6572-52f7-4d6b-a2e5-2b359a8749bd admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:51.901566 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb631a6-3b27-4391-8e89-03b2c559914a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:51.971687 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ef6572-52f7-4d6b-a2e5-2b359a8749bd admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:51.977067 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39f402c-393c-4e6c-9610-724c9b4511c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:51.977722 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f402c-393c-4e6c-9610-724c9b4511c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:51.978006 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f402c-393c-4e6c-9610-724c9b4511c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:51.978281 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f402c-393c-4e6c-9610-724c9b4511c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:51.979121 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39f402c-393c-4e6c-9610-724c9b4511c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:51.981250 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39f402c-393c-4e6c-9610-724c9b4511c3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:51.984265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf18ac-b82e-418a-b5b6-c24714485cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:51.984898 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf18ac-b82e-418a-b5b6-c24714485cd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:51.985441 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf18ac-b82e-418a-b5b6-c24714485cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:51.985732 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf18ac-b82e-418a-b5b6-c24714485cd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:51.986625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bf18ac-b82e-418a-b5b6-c24714485cd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:51.988613 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bf18ac-b82e-418a-b5b6-c24714485cd3 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:52.071470 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 961/1916] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:52:51 2022] POST /identity/v3/projects => generated 371 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:52:52.074588 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf18ac-b82e-418a-b5b6-c24714485cd3 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:52.097916 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 956/1917] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:52:51 2022] POST /identity/v3/users => generated 295 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:52:52.099090 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39f402c-393c-4e6c-9610-724c9b4511c3 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:52.110599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39f402c-393c-4e6c-9610-724c9b4511c3 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:52.179088 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf18ac-b82e-418a-b5b6-c24714485cd3 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:52.195645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8727f-38d4-441f-8a68-90b56cc0facc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:52.196356 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8727f-38d4-441f-8a68-90b56cc0facc admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:52.196742 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8727f-38d4-441f-8a68-90b56cc0facc admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:52.197028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8727f-38d4-441f-8a68-90b56cc0facc admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:52.197707 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f8727f-38d4-441f-8a68-90b56cc0facc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:52.201631 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f8727f-38d4-441f-8a68-90b56cc0facc admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:52.235550 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b105315-a6ae-48a3-9add-fee126f1057b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:52.236330 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b105315-a6ae-48a3-9add-fee126f1057b admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:52.236747 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b105315-a6ae-48a3-9add-fee126f1057b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:52.237087 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b105315-a6ae-48a3-9add-fee126f1057b admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:52.240558 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b105315-a6ae-48a3-9add-fee126f1057b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:52.240558 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b105315-a6ae-48a3-9add-fee126f1057b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:52.250991 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 957/1918] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:52:52.260680 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b105315-a6ae-48a3-9add-fee126f1057b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:52.271424 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b105315-a6ae-48a3-9add-fee126f1057b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:52.276924 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fca05e-d08d-42e0-97bb-46a5ba3fa425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:52.277568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fca05e-d08d-42e0-97bb-46a5ba3fa425 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:52.277852 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fca05e-d08d-42e0-97bb-46a5ba3fa425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:52.278127 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fca05e-d08d-42e0-97bb-46a5ba3fa425 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:52.278888 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fca05e-d08d-42e0-97bb-46a5ba3fa425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:52.280510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fca05e-d08d-42e0-97bb-46a5ba3fa425 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:52.295269 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 962/1919] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:52:52 2022] POST /identity/v3/users => generated 292 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:52:52.299437 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8727f-38d4-441f-8a68-90b56cc0facc admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:52.304430 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 958/1920] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:52:52.308855 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fca05e-d08d-42e0-97bb-46a5ba3fa425 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:52.314756 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8727f-38d4-441f-8a68-90b56cc0facc admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:52.320321 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fca05e-d08d-42e0-97bb-46a5ba3fa425 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:52.322254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f58545-2771-4978-b7b6-4dcf3a0e9e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:52.322786 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f58545-2771-4978-b7b6-4dcf3a0e9e73 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:52.323282 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f58545-2771-4978-b7b6-4dcf3a0e9e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:52.323610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f58545-2771-4978-b7b6-4dcf3a0e9e73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:52.324591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f58545-2771-4978-b7b6-4dcf3a0e9e73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:52.326713 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f58545-2771-4978-b7b6-4dcf3a0e9e73 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:52.327453 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de64f71-96a2-4cb5-a59d-373cfbecd073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:52.328335 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de64f71-96a2-4cb5-a59d-373cfbecd073 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:52.328913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de64f71-96a2-4cb5-a59d-373cfbecd073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:52.329292 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de64f71-96a2-4cb5-a59d-373cfbecd073 admin admin] PATH_INFO: `/v3/projects/d3527596a72a41688b19f58d093681a3/users/cd08f6933dc642659eb72219cc6c321b/roles/e033961a9363456a8510c34414dd839e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:52.346183 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 963/1921] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:52:52.347513 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f58545-2771-4978-b7b6-4dcf3a0e9e73 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:52.354257 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de64f71-96a2-4cb5-a59d-373cfbecd073 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3527596a72a41688b19f58d093681a3, user_id=cd08f6933dc642659eb72219cc6c321b, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:52.357313 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de64f71-96a2-4cb5-a59d-373cfbecd073 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:52.361832 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f58545-2771-4978-b7b6-4dcf3a0e9e73 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:52.367033 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5353b-273a-4b8b-9a6c-1bc0e3db4c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:52.367751 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5353b-273a-4b8b-9a6c-1bc0e3db4c01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:52.368341 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5353b-273a-4b8b-9a6c-1bc0e3db4c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:52.368630 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5353b-273a-4b8b-9a6c-1bc0e3db4c01 admin admin] PATH_INFO: `/v3/projects/e7c8bb84a38942fa9ba0680d0fb97031/users/edc66fda91e34a2083ceba934da3dcd6/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:52.388506 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac5353b-273a-4b8b-9a6c-1bc0e3db4c01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7c8bb84a38942fa9ba0680d0fb97031, user_id=edc66fda91e34a2083ceba934da3dcd6, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:52.391947 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac5353b-273a-4b8b-9a6c-1bc0e3db4c01 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:52.408782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:52.488664 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 959/1922] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:52:52 2022] PUT /identity/v3/projects/d3527596a72a41688b19f58d093681a3/users/cd08f6933dc642659eb72219cc6c321b/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:52:52.499239 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c46d8-d8f1-4e64-b499-e9a6c7bcd003 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:52.499627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c46d8-d8f1-4e64-b499-e9a6c7bcd003 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:52.499920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c46d8-d8f1-4e64-b499-e9a6c7bcd003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:52.538015 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6f8c46d8-d8f1-4e64-b499-e9a6c7bcd003 None None] MFA Rules not processed for user `cd08f6933dc642659eb72219cc6c321b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:52.538869 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 964/1923] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:52:52 2022] PUT /identity/v3/projects/e7c8bb84a38942fa9ba0680d0fb97031/users/edc66fda91e34a2083ceba934da3dcd6/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:52.539285 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6f8c46d8-d8f1-4e64-b499-e9a6c7bcd003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:52.547836 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5353b-273a-4b8b-9a6c-1bc0e3db4c01 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:52.621582 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 960/1924] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:52.622421 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5353b-273a-4b8b-9a6c-1bc0e3db4c01 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:52.627474 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c22f5-8acc-4687-9114-2984d262145e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:52.628360 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c22f5-8acc-4687-9114-2984d262145e admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:52.628652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c22f5-8acc-4687-9114-2984d262145e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:52.628931 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c22f5-8acc-4687-9114-2984d262145e admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:52.629635 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c22f5-8acc-4687-9114-2984d262145e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:52.631430 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c22f5-8acc-4687-9114-2984d262145e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:52.632733 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c46d8-d8f1-4e64-b499-e9a6c7bcd003 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:52.635163 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6f8c46d8-d8f1-4e64-b499-e9a6c7bcd003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:52.648531 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 965/1925] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:52:52.652372 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c22f5-8acc-4687-9114-2984d262145e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:52.663003 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c22f5-8acc-4687-9114-2984d262145e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:52.669620 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5d481-f5b0-4673-90c0-3a27484b1ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:52.670684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5d481-f5b0-4673-90c0-3a27484b1ba7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:52.671382 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5d481-f5b0-4673-90c0-3a27484b1ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:52.672020 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5d481-f5b0-4673-90c0-3a27484b1ba7 admin admin] PATH_INFO: `/v3/projects/e7c8bb84a38942fa9ba0680d0fb97031/users/edc66fda91e34a2083ceba934da3dcd6/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:52.674046 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d5d481-f5b0-4673-90c0-3a27484b1ba7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7c8bb84a38942fa9ba0680d0fb97031, user_id=edc66fda91e34a2083ceba934da3dcd6, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:52.677435 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d5d481-f5b0-4673-90c0-3a27484b1ba7 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:52.691311 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 966/1926] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:52:52 2022] PUT /identity/v3/projects/e7c8bb84a38942fa9ba0680d0fb97031/users/edc66fda91e34a2083ceba934da3dcd6/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:52:52.706103 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c46d8-d8f1-4e64-b499-e9a6c7bcd003 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:52.756714 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919aaaa6-5cd6-4e04-9983-9246789b459d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:52.757336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919aaaa6-5cd6-4e04-9983-9246789b459d service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:52.757643 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919aaaa6-5cd6-4e04-9983-9246789b459d service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:52.757915 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919aaaa6-5cd6-4e04-9983-9246789b459d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:52.815953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919aaaa6-5cd6-4e04-9983-9246789b459d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:52.817465 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919aaaa6-5cd6-4e04-9983-9246789b459d service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:52.826707 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 961/1927] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:52 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:52:54.370400 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbd5cc-9c92-4ef1-b5ab-821b55a77334 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:54.370400 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbd5cc-9c92-4ef1-b5ab-821b55a77334 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:54.370884 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbd5cc-9c92-4ef1-b5ab-821b55a77334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:54.415024 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-1bcbd5cc-9c92-4ef1-b5ab-821b55a77334 None None] MFA Rules not processed for user `edc66fda91e34a2083ceba934da3dcd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:52:54.417609 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1bcbd5cc-9c92-4ef1-b5ab-821b55a77334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:54.524379 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 967/1928] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:52:54 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:52:54.630369 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919aaaa6-5cd6-4e04-9983-9246789b459d service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:52:54.632435 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-919aaaa6-5cd6-4e04-9983-9246789b459d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:52:54.644305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919aaaa6-5cd6-4e04-9983-9246789b459d service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:52:54.648150 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61753955-e6ec-4eb3-a5a6-d548b24ad930 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:52:54.648773 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61753955-e6ec-4eb3-a5a6-d548b24ad930 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:52:54.650431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61753955-e6ec-4eb3-a5a6-d548b24ad930 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:52:54.650431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61753955-e6ec-4eb3-a5a6-d548b24ad930 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:52:54.708725 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61753955-e6ec-4eb3-a5a6-d548b24ad930 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:52:54.709960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61753955-e6ec-4eb3-a5a6-d548b24ad930 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:52:54.719997 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 962/1929] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:52:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:52:54.978058 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:55.857792 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:56.909732 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:57.264128 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:57.504030 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:57.961049 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:58.126958 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:58.264009 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:59.064046 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:59.103716 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:59.161072 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:59.244686 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:52:59.909785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:07.114649 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbd5cc-9c92-4ef1-b5ab-821b55a77334 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:07.126929 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbd5cc-9c92-4ef1-b5ab-821b55a77334 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:07.131231 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd5c72-2b70-4bb9-9a01-c7069d34dffd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:07.132496 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd5c72-2b70-4bb9-9a01-c7069d34dffd service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:07.132839 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd5c72-2b70-4bb9-9a01-c7069d34dffd service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:07.135636 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd5c72-2b70-4bb9-9a01-c7069d34dffd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:07.143822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd5c72-2b70-4bb9-9a01-c7069d34dffd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:07.145320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd5c72-2b70-4bb9-9a01-c7069d34dffd service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:07.166057 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 968/1930] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:53:07 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:53:10.156183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=93 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:16.514871 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61753955-e6ec-4eb3-a5a6-d548b24ad930 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:16.516316 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-61753955-e6ec-4eb3-a5a6-d548b24ad930 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:16.587909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61753955-e6ec-4eb3-a5a6-d548b24ad930 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:16.592974 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bda6aa-badc-43e7-b263-777e5af763ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:16.593734 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bda6aa-badc-43e7-b263-777e5af763ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:16.594044 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bda6aa-badc-43e7-b263-777e5af763ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:16.594346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bda6aa-badc-43e7-b263-777e5af763ba admin admin] PATH_INFO: `/v3/users/299aa937a2004ac181685cbc03ab8146` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:16.595571 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bda6aa-badc-43e7-b263-777e5af763ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=299aa937a2004ac181685cbc03ab8146)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:16.597569 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bda6aa-badc-43e7-b263-777e5af763ba admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:16.827814 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-59bda6aa-badc-43e7-b263-777e5af763ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '299aa937a2004ac181685cbc03ab8146'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:16.828395 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-59bda6aa-badc-43e7-b263-777e5af763ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '299aa937a2004ac181685cbc03ab8146'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:16.841282 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-59bda6aa-badc-43e7-b263-777e5af763ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '299aa937a2004ac181685cbc03ab8146'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:16.848184 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-59bda6aa-badc-43e7-b263-777e5af763ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '299aa937a2004ac181685cbc03ab8146'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:16.848852 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-59bda6aa-badc-43e7-b263-777e5af763ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '299aa937a2004ac181685cbc03ab8146'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:16.922580 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 963/1931] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:53:16 2022] DELETE /identity/v3/users/299aa937a2004ac181685cbc03ab8146 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:53:16.927134 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e291c4-ed3b-4cee-82c1-59f01616a6dd None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:16.927638 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e291c4-ed3b-4cee-82c1-59f01616a6dd None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:16.927804 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e291c4-ed3b-4cee-82c1-59f01616a6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:16.946394 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-f5e291c4-ed3b-4cee-82c1-59f01616a6dd None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:16.947610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f5e291c4-ed3b-4cee-82c1-59f01616a6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:17.008979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 969/1932] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:53:17.020325 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bda6aa-badc-43e7-b263-777e5af763ba admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:17.021790 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-59bda6aa-badc-43e7-b263-777e5af763ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:17.079902 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bda6aa-badc-43e7-b263-777e5af763ba admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:17.083358 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b2b5d-ee83-4ef2-89e3-6c54979984d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:17.084006 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b2b5d-ee83-4ef2-89e3-6c54979984d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:17.084287 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b2b5d-ee83-4ef2-89e3-6c54979984d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:17.084562 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b2b5d-ee83-4ef2-89e3-6c54979984d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:17.093959 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b2b5d-ee83-4ef2-89e3-6c54979984d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:17.095374 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b2b5d-ee83-4ef2-89e3-6c54979984d0 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:17.104696 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 964/1933] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:53:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:53:17.586755 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e291c4-ed3b-4cee-82c1-59f01616a6dd None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:17.588349 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f5e291c4-ed3b-4cee-82c1-59f01616a6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:17.601526 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e291c4-ed3b-4cee-82c1-59f01616a6dd None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:17.608070 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9522d3c-cc94-45db-9d93-445e6b5b34d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:17.608932 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9522d3c-cc94-45db-9d93-445e6b5b34d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:17.609406 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9522d3c-cc94-45db-9d93-445e6b5b34d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:17.609857 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9522d3c-cc94-45db-9d93-445e6b5b34d4 admin admin] PATH_INFO: `/v3/projects/95beb7e102724f2d9172f68a61a65c7e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:17.611219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9522d3c-cc94-45db-9d93-445e6b5b34d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95beb7e102724f2d9172f68a61a65c7e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:17.613457 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9522d3c-cc94-45db-9d93-445e6b5b34d4 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:17.764516 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d9522d3c-cc94-45db-9d93-445e6b5b34d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95beb7e102724f2d9172f68a61a65c7e', 'request_id': 'req-d9522d3c-cc94-45db-9d93-445e6b5b34d4'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:17.770344 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d9522d3c-cc94-45db-9d93-445e6b5b34d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95beb7e102724f2d9172f68a61a65c7e', 'request_id': 'req-d9522d3c-cc94-45db-9d93-445e6b5b34d4'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:17.826094 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d9522d3c-cc94-45db-9d93-445e6b5b34d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95beb7e102724f2d9172f68a61a65c7e', 'request_id': 'req-d9522d3c-cc94-45db-9d93-445e6b5b34d4'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:17.827373 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:17.841628 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d9522d3c-cc94-45db-9d93-445e6b5b34d4 admin admin] The token cache is being invalidate because project 95beb7e102724f2d9172f68a61a65c7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:53:17.841628 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d9522d3c-cc94-45db-9d93-445e6b5b34d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:17.841628 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d9522d3c-cc94-45db-9d93-445e6b5b34d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:17.841628 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 970/1934] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:53:17 2022] DELETE /identity/v3/projects/95beb7e102724f2d9172f68a61a65c7e => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:53:17.845209 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b2b5d-ee83-4ef2-89e3-6c54979984d0 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:17.854807 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b2b5d-ee83-4ef2-89e3-6c54979984d0 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:17.873492 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:17.913207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964609b6-b760-4f7b-b763-2e70c0e8a259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:17.913988 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964609b6-b760-4f7b-b763-2e70c0e8a259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:17.914274 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964609b6-b760-4f7b-b763-2e70c0e8a259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:17.914555 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964609b6-b760-4f7b-b763-2e70c0e8a259 admin admin] PATH_INFO: `/v3/users/741730a5c7934c7f9327831a7015c2c7` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:17.915874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964609b6-b760-4f7b-b763-2e70c0e8a259 admin admin] RBAC: Authorizing `identity:delete_user(user_id=741730a5c7934c7f9327831a7015c2c7)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:17.917832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964609b6-b760-4f7b-b763-2e70c0e8a259 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:18.171725 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-964609b6-b760-4f7b-b763-2e70c0e8a259 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '741730a5c7934c7f9327831a7015c2c7'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:18.172457 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-964609b6-b760-4f7b-b763-2e70c0e8a259 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '741730a5c7934c7f9327831a7015c2c7'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:18.185496 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-964609b6-b760-4f7b-b763-2e70c0e8a259 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '741730a5c7934c7f9327831a7015c2c7'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:18.192405 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-964609b6-b760-4f7b-b763-2e70c0e8a259 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '741730a5c7934c7f9327831a7015c2c7'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:18.193076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-964609b6-b760-4f7b-b763-2e70c0e8a259 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '741730a5c7934c7f9327831a7015c2c7'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:18.267173 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 965/1935] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:53:17 2022] DELETE /identity/v3/users/741730a5c7934c7f9327831a7015c2c7 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:53:18.724452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9522d3c-cc94-45db-9d93-445e6b5b34d4 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:18.725969 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d9522d3c-cc94-45db-9d93-445e6b5b34d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:18.740861 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9522d3c-cc94-45db-9d93-445e6b5b34d4 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:18.801479 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb252d-83da-422d-b67f-d46aa0cd35bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:18.802114 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb252d-83da-422d-b67f-d46aa0cd35bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:18.802404 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb252d-83da-422d-b67f-d46aa0cd35bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:18.802679 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb252d-83da-422d-b67f-d46aa0cd35bf admin admin] PATH_INFO: `/v3/projects/7b6a8cb3b4174123963275fa10ad0565` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:18.803862 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eb252d-83da-422d-b67f-d46aa0cd35bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b6a8cb3b4174123963275fa10ad0565)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:18.805782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eb252d-83da-422d-b67f-d46aa0cd35bf admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:18.956692 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-66eb252d-83da-422d-b67f-d46aa0cd35bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b6a8cb3b4174123963275fa10ad0565', 'request_id': 'req-66eb252d-83da-422d-b67f-d46aa0cd35bf'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:18.962077 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-66eb252d-83da-422d-b67f-d46aa0cd35bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b6a8cb3b4174123963275fa10ad0565', 'request_id': 'req-66eb252d-83da-422d-b67f-d46aa0cd35bf'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:19.128287 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-66eb252d-83da-422d-b67f-d46aa0cd35bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b6a8cb3b4174123963275fa10ad0565', 'request_id': 'req-66eb252d-83da-422d-b67f-d46aa0cd35bf'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:19.141156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-66eb252d-83da-422d-b67f-d46aa0cd35bf admin admin] The token cache is being invalidate because project 7b6a8cb3b4174123963275fa10ad0565 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:53:19.141454 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-66eb252d-83da-422d-b67f-d46aa0cd35bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:19.142376 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-66eb252d-83da-422d-b67f-d46aa0cd35bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:19.144177 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 971/1936] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:53:18 2022] DELETE /identity/v3/projects/7b6a8cb3b4174123963275fa10ad0565 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:53:19.148937 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8bcf0-d365-4551-aeda-4eb615901b7c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:19.149784 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8bcf0-d365-4551-aeda-4eb615901b7c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:19.150215 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8bcf0-d365-4551-aeda-4eb615901b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:19.188047 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-aae8bcf0-d365-4551-aeda-4eb615901b7c None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:19.188047 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-aae8bcf0-d365-4551-aeda-4eb615901b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:19.233563 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 966/1937] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:53:19.241186 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a8680-d8f6-45af-b81c-817494261ef7 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:19.241505 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a8680-d8f6-45af-b81c-817494261ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:19.241783 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a8680-d8f6-45af-b81c-817494261ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:19.262645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ca7a8680-d8f6-45af-b81c-817494261ef7 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:19.263893 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ca7a8680-d8f6-45af-b81c-817494261ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:19.271853 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 972/1938] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:53:19.275983 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae8bcf0-d365-4551-aeda-4eb615901b7c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:19.286337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae8bcf0-d365-4551-aeda-4eb615901b7c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:19.290895 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf6765-f644-4815-8e85-89e1a1ddd5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:19.291552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf6765-f644-4815-8e85-89e1a1ddd5a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:19.291840 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf6765-f644-4815-8e85-89e1a1ddd5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:19.292122 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf6765-f644-4815-8e85-89e1a1ddd5a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:19.292818 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf6765-f644-4815-8e85-89e1a1ddd5a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:53:19.293127 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf6765-f644-4815-8e85-89e1a1ddd5a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:19.294729 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf6765-f644-4815-8e85-89e1a1ddd5a3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:19.306229 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 967/1939] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:53:19.310588 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a8680-d8f6-45af-b81c-817494261ef7 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:19.319939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a8680-d8f6-45af-b81c-817494261ef7 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:19.325227 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dccd0ae-bed8-4d2a-9a5c-d43658252863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:19.325873 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccd0ae-bed8-4d2a-9a5c-d43658252863 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:19.326160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccd0ae-bed8-4d2a-9a5c-d43658252863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:19.326439 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccd0ae-bed8-4d2a-9a5c-d43658252863 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:19.327101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dccd0ae-bed8-4d2a-9a5c-d43658252863 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:19.329049 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dccd0ae-bed8-4d2a-9a5c-d43658252863 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:19.421060 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 973/1940] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:53:19 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:53:19.425273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf6765-f644-4815-8e85-89e1a1ddd5a3 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:19.435688 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf6765-f644-4815-8e85-89e1a1ddd5a3 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:19.462597 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:19.492140 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c1a6dc-b082-473c-851c-69de655dd540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:19.492800 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1a6dc-b082-473c-851c-69de655dd540 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:19.493090 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1a6dc-b082-473c-851c-69de655dd540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:19.493367 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1a6dc-b082-473c-851c-69de655dd540 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:19.494041 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c1a6dc-b082-473c-851c-69de655dd540 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:19.496496 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c1a6dc-b082-473c-851c-69de655dd540 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:19.609729 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 968/1941] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:53:19 2022] POST /identity/v3/users => generated 283 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:53:19.613800 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dccd0ae-bed8-4d2a-9a5c-d43658252863 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:19.625891 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dccd0ae-bed8-4d2a-9a5c-d43658252863 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:19.630505 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97290824-2417-4c0d-a43d-a952090eb71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:19.631477 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97290824-2417-4c0d-a43d-a952090eb71d admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:19.631774 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97290824-2417-4c0d-a43d-a952090eb71d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:19.632056 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97290824-2417-4c0d-a43d-a952090eb71d admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:19.632762 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97290824-2417-4c0d-a43d-a952090eb71d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:19.634474 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97290824-2417-4c0d-a43d-a952090eb71d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:19.645685 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 974/1942] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:53:19.650473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c1a6dc-b082-473c-851c-69de655dd540 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:19.660999 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c1a6dc-b082-473c-851c-69de655dd540 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:19.665866 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa12d6-6972-4445-8004-f30538ff64c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:19.670840 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa12d6-6972-4445-8004-f30538ff64c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:19.670840 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa12d6-6972-4445-8004-f30538ff64c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:19.670840 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa12d6-6972-4445-8004-f30538ff64c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:19.670840 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaa12d6-6972-4445-8004-f30538ff64c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:19.670840 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaa12d6-6972-4445-8004-f30538ff64c1 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:19.680583 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 969/1943] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:53:19.685373 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97290824-2417-4c0d-a43d-a952090eb71d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:19.695744 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97290824-2417-4c0d-a43d-a952090eb71d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:19.701320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8e97d-a734-44a4-ad28-2b60162d37fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:19.701999 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8e97d-a734-44a4-ad28-2b60162d37fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:19.702284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8e97d-a734-44a4-ad28-2b60162d37fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:19.702778 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8e97d-a734-44a4-ad28-2b60162d37fc admin admin] PATH_INFO: `/v3/projects/7d08974f94754fe3abc8431a0afcaded/users/883b817d7ea04290bb121ef75a4bf0d8/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:19.720051 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf8e97d-a734-44a4-ad28-2b60162d37fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d08974f94754fe3abc8431a0afcaded, user_id=883b817d7ea04290bb121ef75a4bf0d8, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:19.722840 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf8e97d-a734-44a4-ad28-2b60162d37fc admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:19.803011 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 975/1944] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:53:19 2022] PUT /identity/v3/projects/7d08974f94754fe3abc8431a0afcaded/users/883b817d7ea04290bb121ef75a4bf0d8/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:53:19.815215 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de1b37-0657-4030-86c2-9fc65995a130 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:19.815544 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de1b37-0657-4030-86c2-9fc65995a130 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:19.815826 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de1b37-0657-4030-86c2-9fc65995a130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:19.843150 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a4de1b37-0657-4030-86c2-9fc65995a130 None None] MFA Rules not processed for user `883b817d7ea04290bb121ef75a4bf0d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:19.844490 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a4de1b37-0657-4030-86c2-9fc65995a130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:19.909018 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 970/1945] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:19 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:53:19.913460 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8e97d-a734-44a4-ad28-2b60162d37fc admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:19.935095 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:19.975718 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8e97d-a734-44a4-ad28-2b60162d37fc admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:19.980840 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41115e5-df7e-471a-8c3f-a674d7d782c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:19.981462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41115e5-df7e-471a-8c3f-a674d7d782c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:19.981743 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41115e5-df7e-471a-8c3f-a674d7d782c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:19.982017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41115e5-df7e-471a-8c3f-a674d7d782c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:19.982686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41115e5-df7e-471a-8c3f-a674d7d782c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:19.984629 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41115e5-df7e-471a-8c3f-a674d7d782c0 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:20.129975 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 976/1946] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:53:19 2022] POST /identity/v3/projects => generated 365 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:53:20.134288 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de1b37-0657-4030-86c2-9fc65995a130 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:20.145439 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de1b37-0657-4030-86c2-9fc65995a130 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:20.202332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb0342-d90b-4e76-acb1-34c42f1d9b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:20.202946 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb0342-d90b-4e76-acb1-34c42f1d9b5f admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:20.203238 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb0342-d90b-4e76-acb1-34c42f1d9b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:20.203556 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb0342-d90b-4e76-acb1-34c42f1d9b5f admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:20.204239 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eb0342-d90b-4e76-acb1-34c42f1d9b5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:20.206181 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eb0342-d90b-4e76-acb1-34c42f1d9b5f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:20.305998 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 971/1947] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:53:20 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:53:20.310609 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41115e5-df7e-471a-8c3f-a674d7d782c0 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:20.321307 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41115e5-df7e-471a-8c3f-a674d7d782c0 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:20.326552 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553a7c5b-a9a2-4885-a4aa-45bf7298795b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:20.327215 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a7c5b-a9a2-4885-a4aa-45bf7298795b admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:20.327546 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a7c5b-a9a2-4885-a4aa-45bf7298795b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:20.334001 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a7c5b-a9a2-4885-a4aa-45bf7298795b admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:20.334001 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553a7c5b-a9a2-4885-a4aa-45bf7298795b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:20.334001 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553a7c5b-a9a2-4885-a4aa-45bf7298795b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:20.341937 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 977/1948] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:53:20.346129 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb0342-d90b-4e76-acb1-34c42f1d9b5f admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:20.356986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb0342-d90b-4e76-acb1-34c42f1d9b5f admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:20.361794 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807532c3-9687-4674-9e19-3f1e621dcd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:20.364380 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807532c3-9687-4674-9e19-3f1e621dcd9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:20.364380 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807532c3-9687-4674-9e19-3f1e621dcd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:20.364380 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807532c3-9687-4674-9e19-3f1e621dcd9f admin admin] PATH_INFO: `/v3/projects/40ee2172aa764db78e5d1dd7955f3009/users/04104f741a8946388ddf98e8b3918071/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:20.379643 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807532c3-9687-4674-9e19-3f1e621dcd9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=40ee2172aa764db78e5d1dd7955f3009, user_id=04104f741a8946388ddf98e8b3918071, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:20.384080 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807532c3-9687-4674-9e19-3f1e621dcd9f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:20.488936 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 972/1949] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:53:20 2022] PUT /identity/v3/projects/40ee2172aa764db78e5d1dd7955f3009/users/04104f741a8946388ddf98e8b3918071/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:53:20.492984 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553a7c5b-a9a2-4885-a4aa-45bf7298795b admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:20.503691 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553a7c5b-a9a2-4885-a4aa-45bf7298795b admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:20.566155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4fd45c-069b-45cd-ba3e-477191745756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:20.566702 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fd45c-069b-45cd-ba3e-477191745756 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:20.567067 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fd45c-069b-45cd-ba3e-477191745756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:20.567372 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fd45c-069b-45cd-ba3e-477191745756 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:20.568137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4fd45c-069b-45cd-ba3e-477191745756 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:20.569744 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4fd45c-069b-45cd-ba3e-477191745756 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:20.579743 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 978/1950] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:53:20.583976 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807532c3-9687-4674-9e19-3f1e621dcd9f admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:20.589645 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:20.595284 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807532c3-9687-4674-9e19-3f1e621dcd9f admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:20.600038 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae3e832-a6da-4fd1-a6e5-5f6e461a5725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:20.600720 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3e832-a6da-4fd1-a6e5-5f6e461a5725 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:20.601010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3e832-a6da-4fd1-a6e5-5f6e461a5725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:20.605594 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3e832-a6da-4fd1-a6e5-5f6e461a5725 admin admin] PATH_INFO: `/v3/projects/40ee2172aa764db78e5d1dd7955f3009/users/04104f741a8946388ddf98e8b3918071/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:20.605594 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae3e832-a6da-4fd1-a6e5-5f6e461a5725 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40ee2172aa764db78e5d1dd7955f3009, user_id=04104f741a8946388ddf98e8b3918071, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:20.606620 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae3e832-a6da-4fd1-a6e5-5f6e461a5725 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:20.617540 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 973/1951] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:53:20 2022] PUT /identity/v3/projects/40ee2172aa764db78e5d1dd7955f3009/users/04104f741a8946388ddf98e8b3918071/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:53:20.628449 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90540506-6734-41e4-9870-c683eac4bccb None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:20.628815 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90540506-6734-41e4-9870-c683eac4bccb None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:20.629173 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90540506-6734-41e4-9870-c683eac4bccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:20.659689 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-90540506-6734-41e4-9870-c683eac4bccb None None] MFA Rules not processed for user `04104f741a8946388ddf98e8b3918071`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:20.661139 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-90540506-6734-41e4-9870-c683eac4bccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:20.734156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 979/1952] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:20 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:53:20.746106 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae3e832-a6da-4fd1-a6e5-5f6e461a5725 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:20.747936 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1ae3e832-a6da-4fd1-a6e5-5f6e461a5725 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:20.813651 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae3e832-a6da-4fd1-a6e5-5f6e461a5725 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:20.817958 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd29cfd-2d67-493c-a704-0d276640a41a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:20.818694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd29cfd-2d67-493c-a704-0d276640a41a service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:20.819028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd29cfd-2d67-493c-a704-0d276640a41a service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:20.819310 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd29cfd-2d67-493c-a704-0d276640a41a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:20.838695 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd29cfd-2d67-493c-a704-0d276640a41a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:20.838695 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd29cfd-2d67-493c-a704-0d276640a41a service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:20.848488 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 974/1953] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:53:20 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:53:20.851301 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90540506-6734-41e4-9870-c683eac4bccb None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:20.852732 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-90540506-6734-41e4-9870-c683eac4bccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:20.873167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:20.919878 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90540506-6734-41e4-9870-c683eac4bccb None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:20.924497 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924bc189-0deb-4532-937a-697caa6b76f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:20.925138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bc189-0deb-4532-937a-697caa6b76f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:20.925420 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bc189-0deb-4532-937a-697caa6b76f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:20.925703 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bc189-0deb-4532-937a-697caa6b76f2 admin admin] PATH_INFO: `/v3/users/fac59206b5b54053be9a28970097b8aa` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:20.926866 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924bc189-0deb-4532-937a-697caa6b76f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fac59206b5b54053be9a28970097b8aa)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:20.928831 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924bc189-0deb-4532-937a-697caa6b76f2 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:21.139805 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd29cfd-2d67-493c-a704-0d276640a41a service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:21.139965 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5cd29cfd-2d67-493c-a704-0d276640a41a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:21.156001 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd29cfd-2d67-493c-a704-0d276640a41a service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:21.160864 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a9b2e4-6afc-4112-897f-ab3d7008914c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:21.160864 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9b2e4-6afc-4112-897f-ab3d7008914c service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:21.160864 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9b2e4-6afc-4112-897f-ab3d7008914c service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:21.160864 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9b2e4-6afc-4112-897f-ab3d7008914c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:21.221064 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a9b2e4-6afc-4112-897f-ab3d7008914c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:21.222499 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a9b2e4-6afc-4112-897f-ab3d7008914c service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:21.232726 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 975/1954] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:53:21 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:53:21.275658 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-924bc189-0deb-4532-937a-697caa6b76f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fac59206b5b54053be9a28970097b8aa'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:21.276344 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-924bc189-0deb-4532-937a-697caa6b76f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fac59206b5b54053be9a28970097b8aa'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:21.291478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-924bc189-0deb-4532-937a-697caa6b76f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fac59206b5b54053be9a28970097b8aa'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:21.299511 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-924bc189-0deb-4532-937a-697caa6b76f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fac59206b5b54053be9a28970097b8aa'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:21.300141 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-924bc189-0deb-4532-937a-697caa6b76f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fac59206b5b54053be9a28970097b8aa'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:21.488904 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 980/1955] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:53:20 2022] DELETE /identity/v3/users/fac59206b5b54053be9a28970097b8aa => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:53:21.493563 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5f813-e111-4a02-9b63-6cb35fb8180f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:21.493923 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5f813-e111-4a02-9b63-6cb35fb8180f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:21.494273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5f813-e111-4a02-9b63-6cb35fb8180f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:21.518035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-31b5f813-e111-4a02-9b63-6cb35fb8180f None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:21.519518 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-31b5f813-e111-4a02-9b63-6cb35fb8180f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:21.584272 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 976/1956] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:53:21.596573 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924bc189-0deb-4532-937a-697caa6b76f2 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:21.598021 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-924bc189-0deb-4532-937a-697caa6b76f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:21.664116 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924bc189-0deb-4532-937a-697caa6b76f2 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:21.667885 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b7001-8d08-4aab-8d30-18817390fdd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:21.668511 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b7001-8d08-4aab-8d30-18817390fdd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:21.668797 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b7001-8d08-4aab-8d30-18817390fdd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:21.669085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b7001-8d08-4aab-8d30-18817390fdd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:21.678612 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7b7001-8d08-4aab-8d30-18817390fdd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:21.679874 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7b7001-8d08-4aab-8d30-18817390fdd7 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:21.694529 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 981/1957] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:53:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:53:22.244076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5f813-e111-4a02-9b63-6cb35fb8180f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:22.246158 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-31b5f813-e111-4a02-9b63-6cb35fb8180f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:22.261786 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5f813-e111-4a02-9b63-6cb35fb8180f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:22.266658 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cecec6-ae27-4d83-83f2-1e6eeea93472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:22.267313 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cecec6-ae27-4d83-83f2-1e6eeea93472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:22.267944 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cecec6-ae27-4d83-83f2-1e6eeea93472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:22.268066 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cecec6-ae27-4d83-83f2-1e6eeea93472 admin admin] PATH_INFO: `/v3/projects/03b667b40abe4a52969ce1787c1c0189` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:22.269228 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cecec6-ae27-4d83-83f2-1e6eeea93472 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03b667b40abe4a52969ce1787c1c0189)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:22.271306 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cecec6-ae27-4d83-83f2-1e6eeea93472 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:22.388201 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:22.407247 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c0cecec6-ae27-4d83-83f2-1e6eeea93472 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03b667b40abe4a52969ce1787c1c0189', 'request_id': 'req-c0cecec6-ae27-4d83-83f2-1e6eeea93472'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:22.412841 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c0cecec6-ae27-4d83-83f2-1e6eeea93472 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03b667b40abe4a52969ce1787c1c0189', 'request_id': 'req-c0cecec6-ae27-4d83-83f2-1e6eeea93472'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:22.557192 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c0cecec6-ae27-4d83-83f2-1e6eeea93472 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03b667b40abe4a52969ce1787c1c0189', 'request_id': 'req-c0cecec6-ae27-4d83-83f2-1e6eeea93472'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:22.569423 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c0cecec6-ae27-4d83-83f2-1e6eeea93472 admin admin] The token cache is being invalidate because project 03b667b40abe4a52969ce1787c1c0189 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:53:22.569716 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c0cecec6-ae27-4d83-83f2-1e6eeea93472 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:22.570397 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c0cecec6-ae27-4d83-83f2-1e6eeea93472 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:22.572153 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 977/1958] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:53:22 2022] DELETE /identity/v3/projects/03b667b40abe4a52969ce1787c1c0189 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:53:22.576844 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b7001-8d08-4aab-8d30-18817390fdd7 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:22.586738 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b7001-8d08-4aab-8d30-18817390fdd7 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:22.621300 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:22.644862 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efda786-7a3c-41af-8224-0fe45af5ee92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:22.645431 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efda786-7a3c-41af-8224-0fe45af5ee92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:22.645710 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efda786-7a3c-41af-8224-0fe45af5ee92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:22.645981 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efda786-7a3c-41af-8224-0fe45af5ee92 admin admin] PATH_INFO: `/v3/users/5dd91094a71448b8a6bf44be2c2b1d5b` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:22.647174 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efda786-7a3c-41af-8224-0fe45af5ee92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dd91094a71448b8a6bf44be2c2b1d5b)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:22.649142 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efda786-7a3c-41af-8224-0fe45af5ee92 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:22.896002 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-5efda786-7a3c-41af-8224-0fe45af5ee92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dd91094a71448b8a6bf44be2c2b1d5b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:22.896576 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-5efda786-7a3c-41af-8224-0fe45af5ee92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dd91094a71448b8a6bf44be2c2b1d5b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:22.908526 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-5efda786-7a3c-41af-8224-0fe45af5ee92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dd91094a71448b8a6bf44be2c2b1d5b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:22.916040 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-5efda786-7a3c-41af-8224-0fe45af5ee92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dd91094a71448b8a6bf44be2c2b1d5b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:22.916732 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-5efda786-7a3c-41af-8224-0fe45af5ee92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dd91094a71448b8a6bf44be2c2b1d5b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:23.022549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 982/1959] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:53:22 2022] DELETE /identity/v3/users/5dd91094a71448b8a6bf44be2c2b1d5b => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:53:23.509208 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cecec6-ae27-4d83-83f2-1e6eeea93472 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:23.511110 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c0cecec6-ae27-4d83-83f2-1e6eeea93472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:23.533142 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cecec6-ae27-4d83-83f2-1e6eeea93472 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:23.601153 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb06e8f-a9b6-48cc-91f6-323c2e2d4683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:23.601806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb06e8f-a9b6-48cc-91f6-323c2e2d4683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:23.602091 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb06e8f-a9b6-48cc-91f6-323c2e2d4683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:23.602371 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb06e8f-a9b6-48cc-91f6-323c2e2d4683 admin admin] PATH_INFO: `/v3/projects/6a4b82827112444394e3808b69b20573` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:23.603585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb06e8f-a9b6-48cc-91f6-323c2e2d4683 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a4b82827112444394e3808b69b20573)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:23.606120 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb06e8f-a9b6-48cc-91f6-323c2e2d4683 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:23.726518 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8bb06e8f-a9b6-48cc-91f6-323c2e2d4683 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a4b82827112444394e3808b69b20573', 'request_id': 'req-8bb06e8f-a9b6-48cc-91f6-323c2e2d4683'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:23.732574 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8bb06e8f-a9b6-48cc-91f6-323c2e2d4683 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a4b82827112444394e3808b69b20573', 'request_id': 'req-8bb06e8f-a9b6-48cc-91f6-323c2e2d4683'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:23.832125 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8bb06e8f-a9b6-48cc-91f6-323c2e2d4683 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a4b82827112444394e3808b69b20573', 'request_id': 'req-8bb06e8f-a9b6-48cc-91f6-323c2e2d4683'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:23.846799 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8bb06e8f-a9b6-48cc-91f6-323c2e2d4683 admin admin] The token cache is being invalidate because project 6a4b82827112444394e3808b69b20573 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:53:23.847227 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8bb06e8f-a9b6-48cc-91f6-323c2e2d4683 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:23.848010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8bb06e8f-a9b6-48cc-91f6-323c2e2d4683 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:23.849808 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 978/1960] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:53:23 2022] DELETE /identity/v3/projects/6a4b82827112444394e3808b69b20573 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:53:23.854914 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a953d-c5b6-4fa0-9f51-d922865b66d6 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:23.855427 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a953d-c5b6-4fa0-9f51-d922865b66d6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:23.855567 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a953d-c5b6-4fa0-9f51-d922865b66d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:23.875603 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-4c7a953d-c5b6-4fa0-9f51-d922865b66d6 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:23.876954 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4c7a953d-c5b6-4fa0-9f51-d922865b66d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:23.946109 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 983/1961] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:53:23.949805 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa2815-b372-4c8e-aeb8-264690734ca5 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:23.950232 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa2815-b372-4c8e-aeb8-264690734ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:23.950534 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa2815-b372-4c8e-aeb8-264690734ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:23.970485 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-66fa2815-b372-4c8e-aeb8-264690734ca5 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:23.971751 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-66fa2815-b372-4c8e-aeb8-264690734ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:23.980482 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 979/1962] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:53:23.984794 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a953d-c5b6-4fa0-9f51-d922865b66d6 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:23.994623 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a953d-c5b6-4fa0-9f51-d922865b66d6 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:24.000845 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109b06d-b092-412b-bf53-fc0da15e3b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:24.001467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109b06d-b092-412b-bf53-fc0da15e3b13 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:24.001751 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109b06d-b092-412b-bf53-fc0da15e3b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:24.002030 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109b06d-b092-412b-bf53-fc0da15e3b13 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:24.002726 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b109b06d-b092-412b-bf53-fc0da15e3b13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:53:24.003035 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b109b06d-b092-412b-bf53-fc0da15e3b13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:24.004773 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b109b06d-b092-412b-bf53-fc0da15e3b13 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:24.016934 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 984/1963] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:53:24.021706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa2815-b372-4c8e-aeb8-264690734ca5 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:24.032983 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa2815-b372-4c8e-aeb8-264690734ca5 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:24.038279 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50ad5f-0702-40bd-b5e7-676ee4bebb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:24.039009 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50ad5f-0702-40bd-b5e7-676ee4bebb72 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:24.039329 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50ad5f-0702-40bd-b5e7-676ee4bebb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:24.039776 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50ad5f-0702-40bd-b5e7-676ee4bebb72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:24.040880 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50ad5f-0702-40bd-b5e7-676ee4bebb72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:24.043932 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50ad5f-0702-40bd-b5e7-676ee4bebb72 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:24.576263 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 980/1964] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:53:24 2022] POST /identity/v3/projects => generated 377 bytes in 552 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:53:24.577455 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109b06d-b092-412b-bf53-fc0da15e3b13 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:24.588032 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109b06d-b092-412b-bf53-fc0da15e3b13 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:24.643522 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:24.652675 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a7a893-2327-47d7-90db-630d528c9604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:24.653306 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a7a893-2327-47d7-90db-630d528c9604 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:24.653616 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a7a893-2327-47d7-90db-630d528c9604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:24.653973 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a7a893-2327-47d7-90db-630d528c9604 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:24.654658 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a7a893-2327-47d7-90db-630d528c9604 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:24.656678 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a7a893-2327-47d7-90db-630d528c9604 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:24.788729 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 985/1965] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:53:24 2022] POST /identity/v3/users => generated 289 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:53:24.789479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50ad5f-0702-40bd-b5e7-676ee4bebb72 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:24.800785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50ad5f-0702-40bd-b5e7-676ee4bebb72 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:24.805640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057318d3-5575-4e52-b7c4-da12f86c4bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:24.806263 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057318d3-5575-4e52-b7c4-da12f86c4bda admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:24.806548 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057318d3-5575-4e52-b7c4-da12f86c4bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:24.806855 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057318d3-5575-4e52-b7c4-da12f86c4bda admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:24.807620 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057318d3-5575-4e52-b7c4-da12f86c4bda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:24.809351 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057318d3-5575-4e52-b7c4-da12f86c4bda admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:24.819641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 981/1966] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:53:24.823759 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a7a893-2327-47d7-90db-630d528c9604 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:24.834453 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a7a893-2327-47d7-90db-630d528c9604 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:24.839251 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2159c8-3577-4cc2-854a-77b601456961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:24.840408 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2159c8-3577-4cc2-854a-77b601456961 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:24.840692 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2159c8-3577-4cc2-854a-77b601456961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:24.840968 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2159c8-3577-4cc2-854a-77b601456961 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:24.841664 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2159c8-3577-4cc2-854a-77b601456961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:24.843252 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2159c8-3577-4cc2-854a-77b601456961 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:24.854106 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 986/1967] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:53:24.858870 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057318d3-5575-4e52-b7c4-da12f86c4bda admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:24.865381 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:24.869139 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057318d3-5575-4e52-b7c4-da12f86c4bda admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:24.873768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d39fa-338e-4b33-b3c4-141e89a89a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:24.874431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d39fa-338e-4b33-b3c4-141e89a89a4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:24.874721 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d39fa-338e-4b33-b3c4-141e89a89a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:24.875002 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d39fa-338e-4b33-b3c4-141e89a89a4e admin admin] PATH_INFO: `/v3/projects/abae4f3354b84328a96144d093dd17be/users/1e12980e507e4b148561ecf82580f377/roles/e033961a9363456a8510c34414dd839e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:24.890109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d39fa-338e-4b33-b3c4-141e89a89a4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=abae4f3354b84328a96144d093dd17be, user_id=1e12980e507e4b148561ecf82580f377, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:24.892956 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d39fa-338e-4b33-b3c4-141e89a89a4e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:25.043588 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 982/1968] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:53:24 2022] PUT /identity/v3/projects/abae4f3354b84328a96144d093dd17be/users/1e12980e507e4b148561ecf82580f377/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:53:25.055093 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fda2e9-ba9e-4e34-98d8-dd1fa034b068 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:25.055610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fda2e9-ba9e-4e34-98d8-dd1fa034b068 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:25.055744 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fda2e9-ba9e-4e34-98d8-dd1fa034b068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:25.083800 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-24fda2e9-ba9e-4e34-98d8-dd1fa034b068 None None] MFA Rules not processed for user `1e12980e507e4b148561ecf82580f377`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:25.086583 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-24fda2e9-ba9e-4e34-98d8-dd1fa034b068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:25.149209 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 987/1969] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:53:25.154277 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d39fa-338e-4b33-b3c4-141e89a89a4e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:25.216572 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d39fa-338e-4b33-b3c4-141e89a89a4e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:25.221320 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0680d2b3-744c-4db1-90bd-b597e4739945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:25.221941 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680d2b3-744c-4db1-90bd-b597e4739945 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:25.222232 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680d2b3-744c-4db1-90bd-b597e4739945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:25.222536 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680d2b3-744c-4db1-90bd-b597e4739945 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:25.223214 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0680d2b3-744c-4db1-90bd-b597e4739945 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:25.225195 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0680d2b3-744c-4db1-90bd-b597e4739945 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:25.315758 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 983/1970] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:53:25 2022] POST /identity/v3/projects => generated 375 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:53:25.319828 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fda2e9-ba9e-4e34-98d8-dd1fa034b068 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:25.331816 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fda2e9-ba9e-4e34-98d8-dd1fa034b068 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:25.389541 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527ed50-470a-4e37-a189-098eb5baa8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:25.390198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527ed50-470a-4e37-a189-098eb5baa8d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:25.390478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527ed50-470a-4e37-a189-098eb5baa8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:25.390749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527ed50-470a-4e37-a189-098eb5baa8d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:25.391526 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b527ed50-470a-4e37-a189-098eb5baa8d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:25.393436 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b527ed50-470a-4e37-a189-098eb5baa8d2 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:25.519285 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 988/1971] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:53:25 2022] POST /identity/v3/users => generated 294 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:53:25.522737 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0680d2b3-744c-4db1-90bd-b597e4739945 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:25.532912 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0680d2b3-744c-4db1-90bd-b597e4739945 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:25.537368 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f6eac-777c-44bb-b684-920363b74c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:25.538009 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f6eac-777c-44bb-b684-920363b74c4b admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:25.538295 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f6eac-777c-44bb-b684-920363b74c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:25.538576 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f6eac-777c-44bb-b684-920363b74c4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:25.539339 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f6eac-777c-44bb-b684-920363b74c4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:25.540953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f6eac-777c-44bb-b684-920363b74c4b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:25.551894 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 984/1972] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:53:25.556690 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527ed50-470a-4e37-a189-098eb5baa8d2 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:25.572905 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527ed50-470a-4e37-a189-098eb5baa8d2 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:25.572905 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24702cf2-04d4-4297-b56c-db0b81f5ab60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:25.572905 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24702cf2-04d4-4297-b56c-db0b81f5ab60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:25.573263 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24702cf2-04d4-4297-b56c-db0b81f5ab60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:25.573401 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24702cf2-04d4-4297-b56c-db0b81f5ab60 admin admin] PATH_INFO: `/v3/projects/b3623655d8da453183a7678ea072564e/users/71cbf8bc9cee4dc6bde3b4d886224c12/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:25.591145 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24702cf2-04d4-4297-b56c-db0b81f5ab60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3623655d8da453183a7678ea072564e, user_id=71cbf8bc9cee4dc6bde3b4d886224c12, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:25.593884 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24702cf2-04d4-4297-b56c-db0b81f5ab60 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:25.798081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 989/1973] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:53:25 2022] PUT /identity/v3/projects/b3623655d8da453183a7678ea072564e/users/71cbf8bc9cee4dc6bde3b4d886224c12/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:53:25.802205 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f6eac-777c-44bb-b684-920363b74c4b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:25.812515 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f6eac-777c-44bb-b684-920363b74c4b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:25.869596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b7a7b-d932-4870-b196-eaac239a589c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:25.870143 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b7a7b-d932-4870-b196-eaac239a589c admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:25.870429 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b7a7b-d932-4870-b196-eaac239a589c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:25.870715 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b7a7b-d932-4870-b196-eaac239a589c admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:25.871459 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b7a7b-d932-4870-b196-eaac239a589c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:25.873067 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b7a7b-d932-4870-b196-eaac239a589c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:25.883200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 985/1974] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:53:25.888027 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24702cf2-04d4-4297-b56c-db0b81f5ab60 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:25.899376 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24702cf2-04d4-4297-b56c-db0b81f5ab60 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:25.905762 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be652a4-9d0f-4068-9111-15ae032eac6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:25.906554 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be652a4-9d0f-4068-9111-15ae032eac6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:25.906892 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be652a4-9d0f-4068-9111-15ae032eac6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:25.907170 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be652a4-9d0f-4068-9111-15ae032eac6c admin admin] PATH_INFO: `/v3/projects/b3623655d8da453183a7678ea072564e/users/71cbf8bc9cee4dc6bde3b4d886224c12/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:25.909477 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be652a4-9d0f-4068-9111-15ae032eac6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3623655d8da453183a7678ea072564e, user_id=71cbf8bc9cee4dc6bde3b4d886224c12, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:25.912217 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be652a4-9d0f-4068-9111-15ae032eac6c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:25.923279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 990/1975] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:53:25 2022] PUT /identity/v3/projects/b3623655d8da453183a7678ea072564e/users/71cbf8bc9cee4dc6bde3b4d886224c12/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:53:25.933947 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021d44e-e9e5-4ec2-8508-df7048b4af16 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:25.934284 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021d44e-e9e5-4ec2-8508-df7048b4af16 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:25.934569 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021d44e-e9e5-4ec2-8508-df7048b4af16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:25.967044 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6021d44e-e9e5-4ec2-8508-df7048b4af16 None None] MFA Rules not processed for user `71cbf8bc9cee4dc6bde3b4d886224c12`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:25.968297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6021d44e-e9e5-4ec2-8508-df7048b4af16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:26.043006 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 986/1976] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:25 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:53:26.054227 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be652a4-9d0f-4068-9111-15ae032eac6c admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:26.055954 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1be652a4-9d0f-4068-9111-15ae032eac6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:26.115027 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be652a4-9d0f-4068-9111-15ae032eac6c admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:26.118979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f87b7b-c922-4bba-82fa-f2736d8e807c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:26.119694 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f87b7b-c922-4bba-82fa-f2736d8e807c service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:26.119927 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f87b7b-c922-4bba-82fa-f2736d8e807c service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:26.120212 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f87b7b-c922-4bba-82fa-f2736d8e807c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:26.176708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f87b7b-c922-4bba-82fa-f2736d8e807c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:26.178141 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f87b7b-c922-4bba-82fa-f2736d8e807c service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:26.188648 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 991/1977] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:53:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:53:26.346909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021d44e-e9e5-4ec2-8508-df7048b4af16 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:26.348374 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6021d44e-e9e5-4ec2-8508-df7048b4af16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:26.412367 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021d44e-e9e5-4ec2-8508-df7048b4af16 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:26.416839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43ffb8-7bbe-4657-be34-b63c6810683a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:26.417486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43ffb8-7bbe-4657-be34-b63c6810683a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:26.417851 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43ffb8-7bbe-4657-be34-b63c6810683a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:26.418131 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43ffb8-7bbe-4657-be34-b63c6810683a admin admin] PATH_INFO: `/v3/users/883b817d7ea04290bb121ef75a4bf0d8` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:26.419303 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d43ffb8-7bbe-4657-be34-b63c6810683a admin admin] RBAC: Authorizing `identity:delete_user(user_id=883b817d7ea04290bb121ef75a4bf0d8)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:26.421281 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d43ffb8-7bbe-4657-be34-b63c6810683a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:26.751652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9d43ffb8-7bbe-4657-be34-b63c6810683a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '883b817d7ea04290bb121ef75a4bf0d8'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:26.752350 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9d43ffb8-7bbe-4657-be34-b63c6810683a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '883b817d7ea04290bb121ef75a4bf0d8'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:26.767454 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9d43ffb8-7bbe-4657-be34-b63c6810683a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '883b817d7ea04290bb121ef75a4bf0d8'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:26.775326 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9d43ffb8-7bbe-4657-be34-b63c6810683a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '883b817d7ea04290bb121ef75a4bf0d8'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:26.776101 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9d43ffb8-7bbe-4657-be34-b63c6810683a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '883b817d7ea04290bb121ef75a4bf0d8'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:26.935167 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 987/1978] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:53:26 2022] DELETE /identity/v3/users/883b817d7ea04290bb121ef75a4bf0d8 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:53:26.939667 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56459e-05c8-4881-a1c7-fad9c823f542 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:26.939965 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56459e-05c8-4881-a1c7-fad9c823f542 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:26.940248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56459e-05c8-4881-a1c7-fad9c823f542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:26.958435 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:26.960347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ff56459e-05c8-4881-a1c7-fad9c823f542 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:26.961831 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ff56459e-05c8-4881-a1c7-fad9c823f542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:27.016787 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43ffb8-7bbe-4657-be34-b63c6810683a admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:27.019907 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9d43ffb8-7bbe-4657-be34-b63c6810683a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:27.045065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 992/1979] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:53:27.051556 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56459e-05c8-4881-a1c7-fad9c823f542 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:27.053478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ff56459e-05c8-4881-a1c7-fad9c823f542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:27.054191 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43ffb8-7bbe-4657-be34-b63c6810683a admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:27.056515 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:27.062499 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c580bd-eb89-487c-9f45-068be660964e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:27.063291 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c580bd-eb89-487c-9f45-068be660964e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:27.065229 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c580bd-eb89-487c-9f45-068be660964e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:27.065229 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c580bd-eb89-487c-9f45-068be660964e admin admin] PATH_INFO: `/v3/users/cd08f6933dc642659eb72219cc6c321b` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:27.065229 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c580bd-eb89-487c-9f45-068be660964e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd08f6933dc642659eb72219cc6c321b)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:27.067750 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c580bd-eb89-487c-9f45-068be660964e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:27.122572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56459e-05c8-4881-a1c7-fad9c823f542 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:27.126361 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c11f9a-e83c-4527-9d83-0238fd2836f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:27.126976 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c11f9a-e83c-4527-9d83-0238fd2836f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:27.127260 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c11f9a-e83c-4527-9d83-0238fd2836f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:27.127567 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c11f9a-e83c-4527-9d83-0238fd2836f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:27.137402 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c11f9a-e83c-4527-9d83-0238fd2836f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:27.157123 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c11f9a-e83c-4527-9d83-0238fd2836f6 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:27.157123 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 993/1980] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:53:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:53:27.486858 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-57c580bd-eb89-487c-9f45-068be660964e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd08f6933dc642659eb72219cc6c321b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:27.487579 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-57c580bd-eb89-487c-9f45-068be660964e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd08f6933dc642659eb72219cc6c321b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:27.501005 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-57c580bd-eb89-487c-9f45-068be660964e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd08f6933dc642659eb72219cc6c321b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:27.508789 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-57c580bd-eb89-487c-9f45-068be660964e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd08f6933dc642659eb72219cc6c321b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:27.509523 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-57c580bd-eb89-487c-9f45-068be660964e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd08f6933dc642659eb72219cc6c321b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:27.628194 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 988/1981] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:53:27 2022] DELETE /identity/v3/users/cd08f6933dc642659eb72219cc6c321b => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:53:27.629381 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6414c-4d0b-4f80-9698-64b1e673a307 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:27.629675 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6414c-4d0b-4f80-9698-64b1e673a307 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:27.629957 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6414c-4d0b-4f80-9698-64b1e673a307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:27.650005 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-81d6414c-4d0b-4f80-9698-64b1e673a307 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:27.651238 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-81d6414c-4d0b-4f80-9698-64b1e673a307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:27.710075 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 994/1982] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:53:27.720924 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c580bd-eb89-487c-9f45-068be660964e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:27.722353 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-57c580bd-eb89-487c-9f45-068be660964e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:27.780375 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c580bd-eb89-487c-9f45-068be660964e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:27.783969 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807e36e-bb43-48cb-9117-4f6bc781c876 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:27.784869 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807e36e-bb43-48cb-9117-4f6bc781c876 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:27.785152 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807e36e-bb43-48cb-9117-4f6bc781c876 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:27.785429 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807e36e-bb43-48cb-9117-4f6bc781c876 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:27.805283 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6414c-4d0b-4f80-9698-64b1e673a307 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:27.805283 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-81d6414c-4d0b-4f80-9698-64b1e673a307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:27.805568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4807e36e-bb43-48cb-9117-4f6bc781c876 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:27.805568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4807e36e-bb43-48cb-9117-4f6bc781c876 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:27.809121 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6414c-4d0b-4f80-9698-64b1e673a307 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:27.811112 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 989/1983] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:53:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:53:27.814129 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5945f39-aea5-451f-8470-4dad501d6e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:27.815082 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5945f39-aea5-451f-8470-4dad501d6e77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:27.815200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5945f39-aea5-451f-8470-4dad501d6e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:27.815420 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5945f39-aea5-451f-8470-4dad501d6e77 admin admin] PATH_INFO: `/v3/projects/7d08974f94754fe3abc8431a0afcaded` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:27.816619 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5945f39-aea5-451f-8470-4dad501d6e77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d08974f94754fe3abc8431a0afcaded)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:27.818702 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5945f39-aea5-451f-8470-4dad501d6e77 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:27.969970 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c5945f39-aea5-451f-8470-4dad501d6e77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d08974f94754fe3abc8431a0afcaded', 'request_id': 'req-c5945f39-aea5-451f-8470-4dad501d6e77'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:27.975616 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c5945f39-aea5-451f-8470-4dad501d6e77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d08974f94754fe3abc8431a0afcaded', 'request_id': 'req-c5945f39-aea5-451f-8470-4dad501d6e77'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:28.084051 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c5945f39-aea5-451f-8470-4dad501d6e77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d08974f94754fe3abc8431a0afcaded', 'request_id': 'req-c5945f39-aea5-451f-8470-4dad501d6e77'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:28.097585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c5945f39-aea5-451f-8470-4dad501d6e77 admin admin] The token cache is being invalidate because project 7d08974f94754fe3abc8431a0afcaded was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:53:28.098022 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c5945f39-aea5-451f-8470-4dad501d6e77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:28.098712 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c5945f39-aea5-451f-8470-4dad501d6e77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:28.103529 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 995/1984] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:53:27 2022] DELETE /identity/v3/projects/7d08974f94754fe3abc8431a0afcaded => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:53:28.104470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807e36e-bb43-48cb-9117-4f6bc781c876 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:28.115593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807e36e-bb43-48cb-9117-4f6bc781c876 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:28.181173 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34729cf-39e8-46e4-b41f-5dbd62170fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:28.181932 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34729cf-39e8-46e4-b41f-5dbd62170fe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:28.182292 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34729cf-39e8-46e4-b41f-5dbd62170fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:28.182672 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34729cf-39e8-46e4-b41f-5dbd62170fe9 admin admin] PATH_INFO: `/v3/users/04104f741a8946388ddf98e8b3918071` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:28.183944 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34729cf-39e8-46e4-b41f-5dbd62170fe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04104f741a8946388ddf98e8b3918071)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:28.185912 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34729cf-39e8-46e4-b41f-5dbd62170fe9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:28.348304 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5945f39-aea5-451f-8470-4dad501d6e77 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:28.349749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c5945f39-aea5-451f-8470-4dad501d6e77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:28.366613 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5945f39-aea5-451f-8470-4dad501d6e77 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:28.372019 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48eca9-a35c-400c-baa0-cdda564eb95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:28.372780 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48eca9-a35c-400c-baa0-cdda564eb95b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:28.373071 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48eca9-a35c-400c-baa0-cdda564eb95b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:28.373329 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48eca9-a35c-400c-baa0-cdda564eb95b admin admin] PATH_INFO: `/v3/projects/d3527596a72a41688b19f58d093681a3` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:28.374530 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd48eca9-a35c-400c-baa0-cdda564eb95b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3527596a72a41688b19f58d093681a3)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:28.383010 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd48eca9-a35c-400c-baa0-cdda564eb95b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:28.434734 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b34729cf-39e8-46e4-b41f-5dbd62170fe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04104f741a8946388ddf98e8b3918071'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:28.435416 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b34729cf-39e8-46e4-b41f-5dbd62170fe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04104f741a8946388ddf98e8b3918071'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:28.447183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b34729cf-39e8-46e4-b41f-5dbd62170fe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04104f741a8946388ddf98e8b3918071'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:28.454832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b34729cf-39e8-46e4-b41f-5dbd62170fe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04104f741a8946388ddf98e8b3918071'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:28.455531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b34729cf-39e8-46e4-b41f-5dbd62170fe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04104f741a8946388ddf98e8b3918071'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:28.518350 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-dd48eca9-a35c-400c-baa0-cdda564eb95b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3527596a72a41688b19f58d093681a3', 'request_id': 'req-dd48eca9-a35c-400c-baa0-cdda564eb95b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:28.524115 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-dd48eca9-a35c-400c-baa0-cdda564eb95b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3527596a72a41688b19f58d093681a3', 'request_id': 'req-dd48eca9-a35c-400c-baa0-cdda564eb95b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:28.603381 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 990/1985] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:53:28 2022] DELETE /identity/v3/users/04104f741a8946388ddf98e8b3918071 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:53:28.764291 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-dd48eca9-a35c-400c-baa0-cdda564eb95b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3527596a72a41688b19f58d093681a3', 'request_id': 'req-dd48eca9-a35c-400c-baa0-cdda564eb95b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:28.776536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-dd48eca9-a35c-400c-baa0-cdda564eb95b admin admin] The token cache is being invalidate because project d3527596a72a41688b19f58d093681a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:53:28.776854 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-dd48eca9-a35c-400c-baa0-cdda564eb95b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:28.777614 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-dd48eca9-a35c-400c-baa0-cdda564eb95b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:28.779199 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 996/1986] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:53:28 2022] DELETE /identity/v3/projects/d3527596a72a41688b19f58d093681a3 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:53:28.785665 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b9ee7-ed59-45e0-8903-62c471c5dd8a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:28.785937 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b9ee7-ed59-45e0-8903-62c471c5dd8a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:28.786254 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b9ee7-ed59-45e0-8903-62c471c5dd8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:28.806331 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-340b9ee7-ed59-45e0-8903-62c471c5dd8a None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:28.807580 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-340b9ee7-ed59-45e0-8903-62c471c5dd8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:28.906643 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 991/1987] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:53:28.919724 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f87c30-f6a7-4bc2-b233-0f2d23a7a824 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:28.920016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f87c30-f6a7-4bc2-b233-0f2d23a7a824 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:28.920297 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f87c30-f6a7-4bc2-b233-0f2d23a7a824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:28.941432 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b6f87c30-f6a7-4bc2-b233-0f2d23a7a824 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:28.942651 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b6f87c30-f6a7-4bc2-b233-0f2d23a7a824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:28.951062 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 997/1988] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:53:28.955337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b9ee7-ed59-45e0-8903-62c471c5dd8a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:28.965280 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b9ee7-ed59-45e0-8903-62c471c5dd8a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:28.970010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e80628-2894-4a32-9625-a41ff3b2b97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:28.970620 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e80628-2894-4a32-9625-a41ff3b2b97a admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:28.970905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e80628-2894-4a32-9625-a41ff3b2b97a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:28.971525 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e80628-2894-4a32-9625-a41ff3b2b97a admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:28.971898 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e80628-2894-4a32-9625-a41ff3b2b97a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:53:28.972207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e80628-2894-4a32-9625-a41ff3b2b97a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:28.973890 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e80628-2894-4a32-9625-a41ff3b2b97a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:28.986209 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 992/1989] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:53:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:53:28.990819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f87c30-f6a7-4bc2-b233-0f2d23a7a824 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:29.000775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f87c30-f6a7-4bc2-b233-0f2d23a7a824 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:29.005501 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd0142c-673b-4056-a62c-6b7254acfab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:29.006127 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0142c-673b-4056-a62c-6b7254acfab3 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:29.006409 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0142c-673b-4056-a62c-6b7254acfab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:29.006683 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0142c-673b-4056-a62c-6b7254acfab3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:29.007819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd0142c-673b-4056-a62c-6b7254acfab3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:29.009729 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd0142c-673b-4056-a62c-6b7254acfab3 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:29.117456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e80628-2894-4a32-9625-a41ff3b2b97a admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:29.119184 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-57e80628-2894-4a32-9625-a41ff3b2b97a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:29.132915 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e80628-2894-4a32-9625-a41ff3b2b97a admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:29.144803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6073ece-82ac-434d-9c5b-da57297fb0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:29.144803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6073ece-82ac-434d-9c5b-da57297fb0e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:29.144803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6073ece-82ac-434d-9c5b-da57297fb0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:29.144803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6073ece-82ac-434d-9c5b-da57297fb0e4 admin admin] PATH_INFO: `/v3/projects/40ee2172aa764db78e5d1dd7955f3009` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:29.151971 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6073ece-82ac-434d-9c5b-da57297fb0e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40ee2172aa764db78e5d1dd7955f3009)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:29.154851 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6073ece-82ac-434d-9c5b-da57297fb0e4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:29.183115 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 998/1990] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:53:28 2022] POST /identity/v3/projects => generated 379 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:53:29.184890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd0142c-673b-4056-a62c-6b7254acfab3 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:29.239524 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:29.250761 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd0142c-673b-4056-a62c-6b7254acfab3 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:29.255684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01edcc1e-9b83-40f8-8d5e-f41704e558a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:29.256317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01edcc1e-9b83-40f8-8d5e-f41704e558a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:29.256606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01edcc1e-9b83-40f8-8d5e-f41704e558a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:29.256897 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01edcc1e-9b83-40f8-8d5e-f41704e558a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:29.257859 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01edcc1e-9b83-40f8-8d5e-f41704e558a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:29.259877 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01edcc1e-9b83-40f8-8d5e-f41704e558a5 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:29.317076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e6073ece-82ac-434d-9c5b-da57297fb0e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40ee2172aa764db78e5d1dd7955f3009', 'request_id': 'req-e6073ece-82ac-434d-9c5b-da57297fb0e4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:29.322358 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e6073ece-82ac-434d-9c5b-da57297fb0e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40ee2172aa764db78e5d1dd7955f3009', 'request_id': 'req-e6073ece-82ac-434d-9c5b-da57297fb0e4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:29.424532 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 999/1991] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:53:29 2022] POST /identity/v3/users => generated 290 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:53:29.425713 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01edcc1e-9b83-40f8-8d5e-f41704e558a5 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:29.435659 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01edcc1e-9b83-40f8-8d5e-f41704e558a5 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:29.496150 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c616051-43e4-4e49-9c0f-fe197bb225fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:29.496808 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c616051-43e4-4e49-9c0f-fe197bb225fa admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:29.497095 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c616051-43e4-4e49-9c0f-fe197bb225fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:29.497384 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c616051-43e4-4e49-9c0f-fe197bb225fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:29.498078 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c616051-43e4-4e49-9c0f-fe197bb225fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:29.500042 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c616051-43e4-4e49-9c0f-fe197bb225fa admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:29.510365 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1000/1992] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:53:29.513370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:29.525049 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e6073ece-82ac-434d-9c5b-da57297fb0e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40ee2172aa764db78e5d1dd7955f3009', 'request_id': 'req-e6073ece-82ac-434d-9c5b-da57297fb0e4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:29.525217 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c616051-43e4-4e49-9c0f-fe197bb225fa admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:29.530081 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e6073ece-82ac-434d-9c5b-da57297fb0e4 admin admin] The token cache is being invalidate because project 40ee2172aa764db78e5d1dd7955f3009 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:53:29.530270 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c616051-43e4-4e49-9c0f-fe197bb225fa admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:29.530417 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e6073ece-82ac-434d-9c5b-da57297fb0e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:29.531021 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e6073ece-82ac-434d-9c5b-da57297fb0e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:29.532566 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 993/1993] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:53:29 2022] DELETE /identity/v3/projects/40ee2172aa764db78e5d1dd7955f3009 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:53:29.537416 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328de85-b375-4e64-bee1-06d318fc2ce4 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:29.537705 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328de85-b375-4e64-bee1-06d318fc2ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:29.537986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328de85-b375-4e64-bee1-06d318fc2ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:29.538153 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b2aa0-a958-45fc-b2aa-34bf1758ed97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:29.538775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b2aa0-a958-45fc-b2aa-34bf1758ed97 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:29.539060 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b2aa0-a958-45fc-b2aa-34bf1758ed97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:29.539336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b2aa0-a958-45fc-b2aa-34bf1758ed97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:29.540087 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b2aa0-a958-45fc-b2aa-34bf1758ed97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:29.541685 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b2aa0-a958-45fc-b2aa-34bf1758ed97 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:29.553231 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1001/1994] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:53:29.557558 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b2aa0-a958-45fc-b2aa-34bf1758ed97 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:29.557739 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-2328de85-b375-4e64-bee1-06d318fc2ce4 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:29.559719 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-532b2aa0-a958-45fc-b2aa-34bf1758ed97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:29.559846 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2328de85-b375-4e64-bee1-06d318fc2ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:29.567802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 994/1995] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:53:29.572439 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b2aa0-a958-45fc-b2aa-34bf1758ed97 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:29.577820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fbef2d-d6e6-487a-8152-542e980326d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:29.578260 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ed7f2-715c-4cdc-b1c5-99b8d6b89fa3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:29.578489 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbef2d-d6e6-487a-8152-542e980326d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:29.578651 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ed7f2-715c-4cdc-b1c5-99b8d6b89fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:29.578761 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbef2d-d6e6-487a-8152-542e980326d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:29.578938 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ed7f2-715c-4cdc-b1c5-99b8d6b89fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:29.579039 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbef2d-d6e6-487a-8152-542e980326d9 admin admin] PATH_INFO: `/v3/projects/e7107f9317c24cc7b972fdc6b8c01322/users/2ae70024479e488c90c6c1b955cb6471/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:29.594166 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fbef2d-d6e6-487a-8152-542e980326d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7107f9317c24cc7b972fdc6b8c01322, user_id=2ae70024479e488c90c6c1b955cb6471, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:29.596954 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fbef2d-d6e6-487a-8152-542e980326d9 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:29.598906 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-9e5ed7f2-715c-4cdc-b1c5-99b8d6b89fa3 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:29.600592 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9e5ed7f2-715c-4cdc-b1c5-99b8d6b89fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:29.610177 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 995/1996] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:53:29.616741 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ed7f2-715c-4cdc-b1c5-99b8d6b89fa3 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:29.626195 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ed7f2-715c-4cdc-b1c5-99b8d6b89fa3 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:29.630856 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae5b11-bbc8-4dab-b4fb-10e4b0eb8d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:29.631528 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae5b11-bbc8-4dab-b4fb-10e4b0eb8d15 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:29.631835 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae5b11-bbc8-4dab-b4fb-10e4b0eb8d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:29.632086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae5b11-bbc8-4dab-b4fb-10e4b0eb8d15 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:29.633033 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ae5b11-bbc8-4dab-b4fb-10e4b0eb8d15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:53:29.633336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ae5b11-bbc8-4dab-b4fb-10e4b0eb8d15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:29.634920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ae5b11-bbc8-4dab-b4fb-10e4b0eb8d15 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:29.647343 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 996/1997] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:53:29.651874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae5b11-bbc8-4dab-b4fb-10e4b0eb8d15 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:29.661993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae5b11-bbc8-4dab-b4fb-10e4b0eb8d15 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:29.666563 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9d232-ca8c-413e-87a3-03afc697e4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:29.667160 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9d232-ca8c-413e-87a3-03afc697e4e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:29.667472 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9d232-ca8c-413e-87a3-03afc697e4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:29.667748 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9d232-ca8c-413e-87a3-03afc697e4e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:29.668443 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d9d232-ca8c-413e-87a3-03afc697e4e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:29.670424 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d9d232-ca8c-413e-87a3-03afc697e4e8 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:29.786547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1002/1998] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:53:29 2022] PUT /identity/v3/projects/e7107f9317c24cc7b972fdc6b8c01322/users/2ae70024479e488c90c6c1b955cb6471/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:53:29.791447 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e769e96-6439-46e5-b60e-bb1610791347 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:29.791740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e769e96-6439-46e5-b60e-bb1610791347 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:29.792024 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e769e96-6439-46e5-b60e-bb1610791347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:29.812574 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-5e769e96-6439-46e5-b60e-bb1610791347 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:29.813873 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5e769e96-6439-46e5-b60e-bb1610791347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:29.877468 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1003/1999] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:53:29.879252 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 997/2000] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:53:29 2022] POST /identity/v3/projects => generated 379 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:53:29.884807 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e769e96-6439-46e5-b60e-bb1610791347 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:29.896498 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9d232-ca8c-413e-87a3-03afc697e4e8 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:29.897336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e769e96-6439-46e5-b60e-bb1610791347 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:29.902214 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b8d9d232-ca8c-413e-87a3-03afc697e4e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:29.971381 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0eb92-eb6e-4bf1-b5f8-67c2b074b506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:29.972065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0eb92-eb6e-4bf1-b5f8-67c2b074b506 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:29.972347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0eb92-eb6e-4bf1-b5f8-67c2b074b506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:29.972623 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0eb92-eb6e-4bf1-b5f8-67c2b074b506 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:29.975094 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad0eb92-eb6e-4bf1-b5f8-67c2b074b506 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:29.976996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad0eb92-eb6e-4bf1-b5f8-67c2b074b506 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:30.001288 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9d232-ca8c-413e-87a3-03afc697e4e8 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:30.005154 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55702a-badb-4f9a-8030-35aa0c1cbc24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:30.005782 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55702a-badb-4f9a-8030-35aa0c1cbc24 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:30.006069 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55702a-badb-4f9a-8030-35aa0c1cbc24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:30.006345 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55702a-badb-4f9a-8030-35aa0c1cbc24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:30.016537 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d55702a-badb-4f9a-8030-35aa0c1cbc24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:30.018029 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d55702a-badb-4f9a-8030-35aa0c1cbc24 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:30.028241 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 998/2001] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:53:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:53:30.101144 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1004/2002] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:53:29 2022] POST /identity/v3/users => generated 290 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:53:30.105354 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55702a-badb-4f9a-8030-35aa0c1cbc24 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:30.115170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55702a-badb-4f9a-8030-35aa0c1cbc24 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:30.119780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8ff1d-2d48-41d9-9951-2a9dd2589635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:30.120384 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8ff1d-2d48-41d9-9951-2a9dd2589635 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:30.120666 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8ff1d-2d48-41d9-9951-2a9dd2589635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:30.120952 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8ff1d-2d48-41d9-9951-2a9dd2589635 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:30.122193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8ff1d-2d48-41d9-9951-2a9dd2589635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:30.123803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8ff1d-2d48-41d9-9951-2a9dd2589635 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:30.134190 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 999/2003] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:53:30.139689 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0eb92-eb6e-4bf1-b5f8-67c2b074b506 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:30.149919 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0eb92-eb6e-4bf1-b5f8-67c2b074b506 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:30.154585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff9310-336d-4b54-af62-71025ee016c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:30.158536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff9310-336d-4b54-af62-71025ee016c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:30.158536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff9310-336d-4b54-af62-71025ee016c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:30.158536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff9310-336d-4b54-af62-71025ee016c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:30.158536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff9310-336d-4b54-af62-71025ee016c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:30.158536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff9310-336d-4b54-af62-71025ee016c0 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:30.167986 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1005/2004] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:53:30.172148 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8ff1d-2d48-41d9-9951-2a9dd2589635 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:30.182057 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8ff1d-2d48-41d9-9951-2a9dd2589635 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:30.187142 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98aa052-3ee0-4b35-ba5b-d735d7d35bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:30.187846 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98aa052-3ee0-4b35-ba5b-d735d7d35bfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:30.188129 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98aa052-3ee0-4b35-ba5b-d735d7d35bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:30.188403 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98aa052-3ee0-4b35-ba5b-d735d7d35bfb admin admin] PATH_INFO: `/v3/projects/c3952799a16144fc81c9f8a21425b514/users/bde1c007b92a43fc8d1ce0dfda2be93c/roles/e033961a9363456a8510c34414dd839e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:30.203301 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98aa052-3ee0-4b35-ba5b-d735d7d35bfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3952799a16144fc81c9f8a21425b514, user_id=bde1c007b92a43fc8d1ce0dfda2be93c, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:30.205988 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98aa052-3ee0-4b35-ba5b-d735d7d35bfb admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:30.369644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1000/2005] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:53:30 2022] PUT /identity/v3/projects/c3952799a16144fc81c9f8a21425b514/users/bde1c007b92a43fc8d1ce0dfda2be93c/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:53:30.382744 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8580284a-e92c-4779-8ca8-10b401bce8eb None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:30.383143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8580284a-e92c-4779-8ca8-10b401bce8eb None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:30.383564 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8580284a-e92c-4779-8ca8-10b401bce8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:30.415890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-8580284a-e92c-4779-8ca8-10b401bce8eb None None] MFA Rules not processed for user `bde1c007b92a43fc8d1ce0dfda2be93c`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:30.417109 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8580284a-e92c-4779-8ca8-10b401bce8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:30.491669 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1006/2006] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:53:30.496351 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98aa052-3ee0-4b35-ba5b-d735d7d35bfb admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:30.561223 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:30.576982 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98aa052-3ee0-4b35-ba5b-d735d7d35bfb admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:30.581168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e810517e-ba06-44cf-bdd3-b0f162a0b52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:30.581806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e810517e-ba06-44cf-bdd3-b0f162a0b52e admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:30.582107 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e810517e-ba06-44cf-bdd3-b0f162a0b52e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:30.582378 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e810517e-ba06-44cf-bdd3-b0f162a0b52e admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:30.583064 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e810517e-ba06-44cf-bdd3-b0f162a0b52e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:30.585162 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e810517e-ba06-44cf-bdd3-b0f162a0b52e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:30.726957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1001/2007] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:53:30 2022] POST /identity/v3/projects => generated 379 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:53:30.731584 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8580284a-e92c-4779-8ca8-10b401bce8eb None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:30.751865 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8580284a-e92c-4779-8ca8-10b401bce8eb None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:30.831149 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41297c6e-e3f2-4d7f-8f62-d5f3ba87170a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:30.836038 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41297c6e-e3f2-4d7f-8f62-d5f3ba87170a admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:30.836038 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41297c6e-e3f2-4d7f-8f62-d5f3ba87170a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:30.836038 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41297c6e-e3f2-4d7f-8f62-d5f3ba87170a admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:30.836038 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41297c6e-e3f2-4d7f-8f62-d5f3ba87170a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:30.836038 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41297c6e-e3f2-4d7f-8f62-d5f3ba87170a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:30.943095 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1007/2008] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:53:30 2022] POST /identity/v3/users => generated 296 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:53:30.950429 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e810517e-ba06-44cf-bdd3-b0f162a0b52e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:30.963314 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e810517e-ba06-44cf-bdd3-b0f162a0b52e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:30.972369 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd8ed3-63f3-42b6-8960-5c628e29c98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:30.972369 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd8ed3-63f3-42b6-8960-5c628e29c98e admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:30.972369 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd8ed3-63f3-42b6-8960-5c628e29c98e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:30.972369 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd8ed3-63f3-42b6-8960-5c628e29c98e admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:30.972730 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbd8ed3-63f3-42b6-8960-5c628e29c98e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:30.972819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbd8ed3-63f3-42b6-8960-5c628e29c98e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:30.985026 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1002/2009] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:53:30.990866 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41297c6e-e3f2-4d7f-8f62-d5f3ba87170a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:31.004637 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41297c6e-e3f2-4d7f-8f62-d5f3ba87170a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:31.075820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7513c5a-3529-4e17-aee7-e7c40221f72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:31.075820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7513c5a-3529-4e17-aee7-e7c40221f72a admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:31.075820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7513c5a-3529-4e17-aee7-e7c40221f72a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:31.075820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7513c5a-3529-4e17-aee7-e7c40221f72a admin admin] PATH_INFO: `/v3/projects/8f93c1a70ce5485e8329a757dd958cf2/users/5406ae5ef8c344d180e5405d66c07c13/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:31.077944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7513c5a-3529-4e17-aee7-e7c40221f72a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f93c1a70ce5485e8329a757dd958cf2, user_id=5406ae5ef8c344d180e5405d66c07c13, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:31.077944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7513c5a-3529-4e17-aee7-e7c40221f72a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:31.181827 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1008/2010] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:53:30 2022] PUT /identity/v3/projects/8f93c1a70ce5485e8329a757dd958cf2/users/5406ae5ef8c344d180e5405d66c07c13/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:53:31.186316 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd8ed3-63f3-42b6-8960-5c628e29c98e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:31.202771 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd8ed3-63f3-42b6-8960-5c628e29c98e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:31.268005 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b75f3-c091-4f61-be86-a0f7ddaf6239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:31.270163 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b75f3-c091-4f61-be86-a0f7ddaf6239 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:31.270163 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b75f3-c091-4f61-be86-a0f7ddaf6239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:31.270163 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b75f3-c091-4f61-be86-a0f7ddaf6239 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:31.270163 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34b75f3-c091-4f61-be86-a0f7ddaf6239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:31.271895 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34b75f3-c091-4f61-be86-a0f7ddaf6239 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:31.283244 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1003/2011] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:53:31 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:53:31.287769 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7513c5a-3529-4e17-aee7-e7c40221f72a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:31.299631 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7513c5a-3529-4e17-aee7-e7c40221f72a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:31.304949 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5fbbff-e91b-460b-958c-a75040822504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:31.305700 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5fbbff-e91b-460b-958c-a75040822504 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:31.306010 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5fbbff-e91b-460b-958c-a75040822504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:31.306284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5fbbff-e91b-460b-958c-a75040822504 admin admin] PATH_INFO: `/v3/projects/8f93c1a70ce5485e8329a757dd958cf2/users/5406ae5ef8c344d180e5405d66c07c13/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:31.308349 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5fbbff-e91b-460b-958c-a75040822504 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f93c1a70ce5485e8329a757dd958cf2, user_id=5406ae5ef8c344d180e5405d66c07c13, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:31.311353 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5fbbff-e91b-460b-958c-a75040822504 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:31.322546 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1009/2012] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:53:31 2022] PUT /identity/v3/projects/8f93c1a70ce5485e8329a757dd958cf2/users/5406ae5ef8c344d180e5405d66c07c13/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:53:31.333442 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1aa45-116c-4a05-b41e-ea88be5fd9ca None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:31.333718 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1aa45-116c-4a05-b41e-ea88be5fd9ca None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:31.334173 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1aa45-116c-4a05-b41e-ea88be5fd9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:31.363218 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-efd1aa45-116c-4a05-b41e-ea88be5fd9ca None None] MFA Rules not processed for user `5406ae5ef8c344d180e5405d66c07c13`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:31.364556 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-efd1aa45-116c-4a05-b41e-ea88be5fd9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:31.441510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1004/2013] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:31 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:53:31.455604 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5fbbff-e91b-460b-958c-a75040822504 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:31.457391 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2e5fbbff-e91b-460b-958c-a75040822504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:31.559820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5fbbff-e91b-460b-958c-a75040822504 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:31.564150 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19919ec3-3b4d-407f-991c-1b73b87310e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:31.564880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19919ec3-3b4d-407f-991c-1b73b87310e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:31.565237 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19919ec3-3b4d-407f-991c-1b73b87310e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:31.565585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19919ec3-3b4d-407f-991c-1b73b87310e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:31.576316 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19919ec3-3b4d-407f-991c-1b73b87310e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:31.578074 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19919ec3-3b4d-407f-991c-1b73b87310e9 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:31.590403 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1010/2014] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:53:31 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:53:32.182396 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd1aa45-116c-4a05-b41e-ea88be5fd9ca None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:32.249691 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd1aa45-116c-4a05-b41e-ea88be5fd9ca None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:32.255253 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d10d14-b523-4091-93c7-2112286424d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:32.257168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:32.257932 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d10d14-b523-4091-93c7-2112286424d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:32.258230 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d10d14-b523-4091-93c7-2112286424d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:32.258577 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d10d14-b523-4091-93c7-2112286424d7 admin admin] PATH_INFO: `/v3/users/bde1c007b92a43fc8d1ce0dfda2be93c` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:32.260170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d10d14-b523-4091-93c7-2112286424d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bde1c007b92a43fc8d1ce0dfda2be93c)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:32.271653 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d10d14-b523-4091-93c7-2112286424d7 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:32.503756 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-34d10d14-b523-4091-93c7-2112286424d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bde1c007b92a43fc8d1ce0dfda2be93c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:32.504525 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-34d10d14-b523-4091-93c7-2112286424d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bde1c007b92a43fc8d1ce0dfda2be93c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:32.518507 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-34d10d14-b523-4091-93c7-2112286424d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bde1c007b92a43fc8d1ce0dfda2be93c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:32.525511 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-34d10d14-b523-4091-93c7-2112286424d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bde1c007b92a43fc8d1ce0dfda2be93c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:32.528272 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-34d10d14-b523-4091-93c7-2112286424d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bde1c007b92a43fc8d1ce0dfda2be93c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:32.622083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1005/2015] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:53:32 2022] DELETE /identity/v3/users/bde1c007b92a43fc8d1ce0dfda2be93c => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:53:32.627197 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65651420-88eb-41fe-ba27-508715312564 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:32.627751 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65651420-88eb-41fe-ba27-508715312564 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:32.627848 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65651420-88eb-41fe-ba27-508715312564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:32.665080 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-65651420-88eb-41fe-ba27-508715312564 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:32.666318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-65651420-88eb-41fe-ba27-508715312564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:32.743363 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1011/2016] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:53:32.752176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d10d14-b523-4091-93c7-2112286424d7 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:32.753604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-34d10d14-b523-4091-93c7-2112286424d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:32.837139 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d10d14-b523-4091-93c7-2112286424d7 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:32.841554 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605db346-81c0-4973-8acb-4254c853e985 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:32.841961 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605db346-81c0-4973-8acb-4254c853e985 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:32.842248 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605db346-81c0-4973-8acb-4254c853e985 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:32.842529 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605db346-81c0-4973-8acb-4254c853e985 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:32.853073 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605db346-81c0-4973-8acb-4254c853e985 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:32.854685 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605db346-81c0-4973-8acb-4254c853e985 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:32.865635 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1006/2017] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:53:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:53:32.895442 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:33.404143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65651420-88eb-41fe-ba27-508715312564 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:33.405680 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-65651420-88eb-41fe-ba27-508715312564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:33.418115 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65651420-88eb-41fe-ba27-508715312564 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:33.422860 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc80df-fd03-4062-8912-4f22f796de3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:33.423552 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc80df-fd03-4062-8912-4f22f796de3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:33.423895 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc80df-fd03-4062-8912-4f22f796de3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:33.424180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc80df-fd03-4062-8912-4f22f796de3d admin admin] PATH_INFO: `/v3/projects/c3952799a16144fc81c9f8a21425b514` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:33.425350 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc80df-fd03-4062-8912-4f22f796de3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3952799a16144fc81c9f8a21425b514)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:33.427305 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc80df-fd03-4062-8912-4f22f796de3d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:33.591450 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9fbc80df-fd03-4062-8912-4f22f796de3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3952799a16144fc81c9f8a21425b514', 'request_id': 'req-9fbc80df-fd03-4062-8912-4f22f796de3d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:33.597681 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9fbc80df-fd03-4062-8912-4f22f796de3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3952799a16144fc81c9f8a21425b514', 'request_id': 'req-9fbc80df-fd03-4062-8912-4f22f796de3d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:33.803913 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9fbc80df-fd03-4062-8912-4f22f796de3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3952799a16144fc81c9f8a21425b514', 'request_id': 'req-9fbc80df-fd03-4062-8912-4f22f796de3d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:33.819346 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9fbc80df-fd03-4062-8912-4f22f796de3d admin admin] The token cache is being invalidate because project c3952799a16144fc81c9f8a21425b514 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:53:33.819874 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9fbc80df-fd03-4062-8912-4f22f796de3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:33.820631 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9fbc80df-fd03-4062-8912-4f22f796de3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:33.822212 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1012/2018] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:53:33 2022] DELETE /identity/v3/projects/c3952799a16144fc81c9f8a21425b514 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:53:33.826506 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605db346-81c0-4973-8acb-4254c853e985 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:33.836986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605db346-81c0-4973-8acb-4254c853e985 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:33.901667 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be3c7db-9a52-4d4b-9780-a9d5b2bcc104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:33.903619 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3c7db-9a52-4d4b-9780-a9d5b2bcc104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:33.903619 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3c7db-9a52-4d4b-9780-a9d5b2bcc104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:33.903619 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3c7db-9a52-4d4b-9780-a9d5b2bcc104 admin admin] PATH_INFO: `/v3/users/5406ae5ef8c344d180e5405d66c07c13` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:33.904644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be3c7db-9a52-4d4b-9780-a9d5b2bcc104 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5406ae5ef8c344d180e5405d66c07c13)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:33.906590 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be3c7db-9a52-4d4b-9780-a9d5b2bcc104 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:34.192349 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:34.216887 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5be3c7db-9a52-4d4b-9780-a9d5b2bcc104 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5406ae5ef8c344d180e5405d66c07c13'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:34.217756 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5be3c7db-9a52-4d4b-9780-a9d5b2bcc104 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5406ae5ef8c344d180e5405d66c07c13'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:34.245460 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5be3c7db-9a52-4d4b-9780-a9d5b2bcc104 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5406ae5ef8c344d180e5405d66c07c13'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:34.253372 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5be3c7db-9a52-4d4b-9780-a9d5b2bcc104 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5406ae5ef8c344d180e5405d66c07c13'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:34.254362 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5be3c7db-9a52-4d4b-9780-a9d5b2bcc104 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5406ae5ef8c344d180e5405d66c07c13'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:34.342090 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:34.390831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1007/2019] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:53:33 2022] DELETE /identity/v3/users/5406ae5ef8c344d180e5405d66c07c13 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:53:35.093400 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc80df-fd03-4062-8912-4f22f796de3d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:35.095708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9fbc80df-fd03-4062-8912-4f22f796de3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:35.119724 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc80df-fd03-4062-8912-4f22f796de3d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:35.209800 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69696ab1-108f-4e3d-bdbb-e52b59759acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:35.210982 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69696ab1-108f-4e3d-bdbb-e52b59759acf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:35.211621 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69696ab1-108f-4e3d-bdbb-e52b59759acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:35.212125 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69696ab1-108f-4e3d-bdbb-e52b59759acf admin admin] PATH_INFO: `/v3/projects/8f93c1a70ce5485e8329a757dd958cf2` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:35.214065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69696ab1-108f-4e3d-bdbb-e52b59759acf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f93c1a70ce5485e8329a757dd958cf2)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:35.225256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69696ab1-108f-4e3d-bdbb-e52b59759acf admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:35.408099 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-69696ab1-108f-4e3d-bdbb-e52b59759acf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f93c1a70ce5485e8329a757dd958cf2', 'request_id': 'req-69696ab1-108f-4e3d-bdbb-e52b59759acf'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:35.413944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-69696ab1-108f-4e3d-bdbb-e52b59759acf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f93c1a70ce5485e8329a757dd958cf2', 'request_id': 'req-69696ab1-108f-4e3d-bdbb-e52b59759acf'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:35.519731 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-69696ab1-108f-4e3d-bdbb-e52b59759acf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f93c1a70ce5485e8329a757dd958cf2', 'request_id': 'req-69696ab1-108f-4e3d-bdbb-e52b59759acf'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:35.544775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-69696ab1-108f-4e3d-bdbb-e52b59759acf admin admin] The token cache is being invalidate because project 8f93c1a70ce5485e8329a757dd958cf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:53:35.545754 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-69696ab1-108f-4e3d-bdbb-e52b59759acf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:35.547550 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-69696ab1-108f-4e3d-bdbb-e52b59759acf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:35.552042 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1013/2020] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:53:35 2022] DELETE /identity/v3/projects/8f93c1a70ce5485e8329a757dd958cf2 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:53:35.557081 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44732bd4-62a6-4588-8492-11c15c1567c9 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:35.557778 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44732bd4-62a6-4588-8492-11c15c1567c9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:35.557778 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44732bd4-62a6-4588-8492-11c15c1567c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:35.585837 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-44732bd4-62a6-4588-8492-11c15c1567c9 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:35.587515 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-44732bd4-62a6-4588-8492-11c15c1567c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:35.685892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1008/2021] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:53:35.695899 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee62781-afd9-42c7-9c98-db5135e32a29 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:35.696492 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee62781-afd9-42c7-9c98-db5135e32a29 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:35.697032 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee62781-afd9-42c7-9c98-db5135e32a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:35.709729 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:35.730334 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-bee62781-afd9-42c7-9c98-db5135e32a29 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:35.733151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-bee62781-afd9-42c7-9c98-db5135e32a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:35.749530 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44732bd4-62a6-4588-8492-11c15c1567c9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:35.760516 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1014/2022] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:53:35.763654 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44732bd4-62a6-4588-8492-11c15c1567c9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:35.770494 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca15d0-c07e-439c-a8c8-eca4b17276cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:35.771444 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca15d0-c07e-439c-a8c8-eca4b17276cf admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:35.771877 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca15d0-c07e-439c-a8c8-eca4b17276cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:35.772313 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca15d0-c07e-439c-a8c8-eca4b17276cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:35.773194 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ca15d0-c07e-439c-a8c8-eca4b17276cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:53:35.773646 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ca15d0-c07e-439c-a8c8-eca4b17276cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:35.775903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ca15d0-c07e-439c-a8c8-eca4b17276cf admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:35.798337 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee62781-afd9-42c7-9c98-db5135e32a29 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:35.804702 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1009/2023] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:53:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:53:35.809236 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee62781-afd9-42c7-9c98-db5135e32a29 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:35.815280 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae42afb-c36b-4b8f-b729-9d0692e9b9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:35.815280 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae42afb-c36b-4b8f-b729-9d0692e9b9ef admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:35.815280 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae42afb-c36b-4b8f-b729-9d0692e9b9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:35.815634 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae42afb-c36b-4b8f-b729-9d0692e9b9ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:35.816300 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae42afb-c36b-4b8f-b729-9d0692e9b9ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:35.818668 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae42afb-c36b-4b8f-b729-9d0692e9b9ef admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:35.936117 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1015/2024] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:53:35 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:53:35.940593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca15d0-c07e-439c-a8c8-eca4b17276cf admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:35.952401 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca15d0-c07e-439c-a8c8-eca4b17276cf admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:36.027337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa982da-c2eb-49e3-a9be-ad72e63b5b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:36.028938 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa982da-c2eb-49e3-a9be-ad72e63b5b9a admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:36.029427 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa982da-c2eb-49e3-a9be-ad72e63b5b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:36.029957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa982da-c2eb-49e3-a9be-ad72e63b5b9a admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:36.031261 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa982da-c2eb-49e3-a9be-ad72e63b5b9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:36.035045 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa982da-c2eb-49e3-a9be-ad72e63b5b9a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:36.148949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1010/2025] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:53:35 2022] POST /identity/v3/users => generated 287 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:53:36.153933 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae42afb-c36b-4b8f-b729-9d0692e9b9ef admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:36.192684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae42afb-c36b-4b8f-b729-9d0692e9b9ef admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:36.216974 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23330b-b850-431e-8886-5a825407fe8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:36.218347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23330b-b850-431e-8886-5a825407fe8e admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:36.219241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23330b-b850-431e-8886-5a825407fe8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:36.219934 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23330b-b850-431e-8886-5a825407fe8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:36.221686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b23330b-b850-431e-8886-5a825407fe8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:36.226740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b23330b-b850-431e-8886-5a825407fe8e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:36.282725 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1016/2026] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:53:36.284229 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa982da-c2eb-49e3-a9be-ad72e63b5b9a admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:36.299706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa982da-c2eb-49e3-a9be-ad72e63b5b9a admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:36.308800 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a74e4e-537a-42a1-9e33-32f590704210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:36.309546 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a74e4e-537a-42a1-9e33-32f590704210 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:36.309831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a74e4e-537a-42a1-9e33-32f590704210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:36.310128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a74e4e-537a-42a1-9e33-32f590704210 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:36.310839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a74e4e-537a-42a1-9e33-32f590704210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:36.312521 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a74e4e-537a-42a1-9e33-32f590704210 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:36.334194 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1011/2027] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:53:36.335840 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23330b-b850-431e-8886-5a825407fe8e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:36.366692 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23330b-b850-431e-8886-5a825407fe8e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:36.377459 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944314a6-2329-4cbc-8a65-e37763d28a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:36.378727 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944314a6-2329-4cbc-8a65-e37763d28a62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:36.379312 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944314a6-2329-4cbc-8a65-e37763d28a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:36.379942 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944314a6-2329-4cbc-8a65-e37763d28a62 admin admin] PATH_INFO: `/v3/projects/a4897a6c07e8482d9bbf88654e04f392/users/eafdad5573674d99ab0010550c948e8b/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:36.418805 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944314a6-2329-4cbc-8a65-e37763d28a62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4897a6c07e8482d9bbf88654e04f392, user_id=eafdad5573674d99ab0010550c948e8b, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:36.430571 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944314a6-2329-4cbc-8a65-e37763d28a62 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:36.502136 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a74e4e-537a-42a1-9e33-32f590704210 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:36.506041 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c0a74e4e-537a-42a1-9e33-32f590704210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:36.609031 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1017/2028] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:53:36 2022] PUT /identity/v3/projects/a4897a6c07e8482d9bbf88654e04f392/users/eafdad5573674d99ab0010550c948e8b/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:53:36.622194 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0eb34-4e24-4d5f-85ca-f99c323ae153 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:36.622967 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0eb34-4e24-4d5f-85ca-f99c323ae153 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:36.623544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0eb34-4e24-4d5f-85ca-f99c323ae153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:36.657339 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:36.690393 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a74e4e-537a-42a1-9e33-32f590704210 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:36.703302 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-2ee0eb34-4e24-4d5f-85ca-f99c323ae153 None None] MFA Rules not processed for user `eafdad5573674d99ab0010550c948e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:36.705878 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2ee0eb34-4e24-4d5f-85ca-f99c323ae153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:36.827283 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774cbd62-9e89-41b1-834d-81dec6649e93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:36.832684 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774cbd62-9e89-41b1-834d-81dec6649e93 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:36.832684 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774cbd62-9e89-41b1-834d-81dec6649e93 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:36.833042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774cbd62-9e89-41b1-834d-81dec6649e93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:36.837275 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-774cbd62-9e89-41b1-834d-81dec6649e93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:36.940748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1018/2029] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:53:36.959734 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0eb34-4e24-4d5f-85ca-f99c323ae153 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:36.963644 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2ee0eb34-4e24-4d5f-85ca-f99c323ae153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:36.991994 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0eb34-4e24-4d5f-85ca-f99c323ae153 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:37.012815 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616cfb37-2b34-43e7-a5a7-fa3f532f9e22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:37.014050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616cfb37-2b34-43e7-a5a7-fa3f532f9e22 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:37.014595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616cfb37-2b34-43e7-a5a7-fa3f532f9e22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:37.015625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616cfb37-2b34-43e7-a5a7-fa3f532f9e22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:37.015802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774cbd62-9e89-41b1-834d-81dec6649e93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:37.017249 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774cbd62-9e89-41b1-834d-81dec6649e93 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:37.030491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1012/2030] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:53:36 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:53:37.070754 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616cfb37-2b34-43e7-a5a7-fa3f532f9e22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:37.079121 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616cfb37-2b34-43e7-a5a7-fa3f532f9e22 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:37.158587 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1019/2031] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:53:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:53:37.434742 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5793f0-8235-4ac3-b015-14d30cc7fdd0 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:37.435260 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5793f0-8235-4ac3-b015-14d30cc7fdd0 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:37.435931 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5793f0-8235-4ac3-b015-14d30cc7fdd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:37.480952 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-aa5793f0-8235-4ac3-b015-14d30cc7fdd0 None None] MFA Rules not processed for user `2ae70024479e488c90c6c1b955cb6471`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:37.482234 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-aa5793f0-8235-4ac3-b015-14d30cc7fdd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:37.576997 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1013/2032] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:53:37.582403 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616cfb37-2b34-43e7-a5a7-fa3f532f9e22 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:37.584796 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-616cfb37-2b34-43e7-a5a7-fa3f532f9e22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:37.670113 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616cfb37-2b34-43e7-a5a7-fa3f532f9e22 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:37.674732 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96037e-5855-41ca-9997-b1ebd01fa8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:37.675297 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96037e-5855-41ca-9997-b1ebd01fa8af admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:37.675610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96037e-5855-41ca-9997-b1ebd01fa8af admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:37.675892 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96037e-5855-41ca-9997-b1ebd01fa8af admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:37.676566 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96037e-5855-41ca-9997-b1ebd01fa8af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:37.679360 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96037e-5855-41ca-9997-b1ebd01fa8af admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:37.845570 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1020/2033] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:53:37 2022] POST /identity/v3/projects => generated 379 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:53:37.847363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5793f0-8235-4ac3-b015-14d30cc7fdd0 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:37.862347 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5793f0-8235-4ac3-b015-14d30cc7fdd0 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:37.976765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:37.979421 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c8b65-5032-4c68-9370-900c32b02029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:37.980693 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c8b65-5032-4c68-9370-900c32b02029 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:37.981797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c8b65-5032-4c68-9370-900c32b02029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:37.982245 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c8b65-5032-4c68-9370-900c32b02029 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:37.983510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74c8b65-5032-4c68-9370-900c32b02029 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:37.985955 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74c8b65-5032-4c68-9370-900c32b02029 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:38.091558 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:38.104178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1014/2034] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:53:37 2022] POST /identity/v3/users => generated 296 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:53:38.109266 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96037e-5855-41ca-9997-b1ebd01fa8af admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:38.121629 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96037e-5855-41ca-9997-b1ebd01fa8af admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:38.127170 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df204c82-20bc-46c0-a63f-6a278620ff87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:38.128158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df204c82-20bc-46c0-a63f-6a278620ff87 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:38.128719 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df204c82-20bc-46c0-a63f-6a278620ff87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:38.129238 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df204c82-20bc-46c0-a63f-6a278620ff87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:38.138453 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df204c82-20bc-46c0-a63f-6a278620ff87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:38.140441 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df204c82-20bc-46c0-a63f-6a278620ff87 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:38.155797 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1021/2035] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:53:38 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:53:38.157712 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c8b65-5032-4c68-9370-900c32b02029 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:38.176157 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c8b65-5032-4c68-9370-900c32b02029 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:38.181401 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b264db-3e71-463a-a34e-40ded2ede29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:38.197131 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b264db-3e71-463a-a34e-40ded2ede29e admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:38.197131 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b264db-3e71-463a-a34e-40ded2ede29e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:38.197131 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b264db-3e71-463a-a34e-40ded2ede29e admin admin] PATH_INFO: `/v3/projects/2bb78824c1344af8a6aea40818681920/users/d46bafbe150a47de957347791d94ed45/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:38.200559 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b264db-3e71-463a-a34e-40ded2ede29e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bb78824c1344af8a6aea40818681920, user_id=d46bafbe150a47de957347791d94ed45, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:38.203274 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b264db-3e71-463a-a34e-40ded2ede29e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:38.330286 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df204c82-20bc-46c0-a63f-6a278620ff87 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:38.331641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1015/2036] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:53:38 2022] PUT /identity/v3/projects/2bb78824c1344af8a6aea40818681920/users/d46bafbe150a47de957347791d94ed45/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:53:38.356745 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df204c82-20bc-46c0-a63f-6a278620ff87 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:38.509194 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2754dc-7880-43e7-ad3a-98bc51b85ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:38.510277 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2754dc-7880-43e7-ad3a-98bc51b85ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:38.510799 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2754dc-7880-43e7-ad3a-98bc51b85ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:38.511312 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2754dc-7880-43e7-ad3a-98bc51b85ed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:38.512909 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2754dc-7880-43e7-ad3a-98bc51b85ed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:38.516042 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2754dc-7880-43e7-ad3a-98bc51b85ed7 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:38.536973 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1022/2037] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:53:38 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:53:38.541214 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b264db-3e71-463a-a34e-40ded2ede29e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:38.553619 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b264db-3e71-463a-a34e-40ded2ede29e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:38.560128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae457ca-61be-47ce-88d7-a2a348c0dd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:38.562583 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae457ca-61be-47ce-88d7-a2a348c0dd5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:38.562583 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae457ca-61be-47ce-88d7-a2a348c0dd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:38.562583 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae457ca-61be-47ce-88d7-a2a348c0dd5c admin admin] PATH_INFO: `/v3/projects/2bb78824c1344af8a6aea40818681920/users/d46bafbe150a47de957347791d94ed45/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:38.563599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae457ca-61be-47ce-88d7-a2a348c0dd5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bb78824c1344af8a6aea40818681920, user_id=d46bafbe150a47de957347791d94ed45, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:38.566345 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae457ca-61be-47ce-88d7-a2a348c0dd5c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:38.578953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1016/2038] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:53:38 2022] PUT /identity/v3/projects/2bb78824c1344af8a6aea40818681920/users/d46bafbe150a47de957347791d94ed45/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:53:40.709697 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:40.739461 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:42.065621 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:42.544672 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:45.490671 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22abc6-938c-4448-af4e-9978a2819c14 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:45.491184 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22abc6-938c-4448-af4e-9978a2819c14 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:45.491510 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22abc6-938c-4448-af4e-9978a2819c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:45.526092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3f22abc6-938c-4448-af4e-9978a2819c14 None None] MFA Rules not processed for user `d46bafbe150a47de957347791d94ed45`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:45.527324 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3f22abc6-938c-4448-af4e-9978a2819c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:45.617523 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1023/2039] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:45 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:53:45.625051 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae457ca-61be-47ce-88d7-a2a348c0dd5c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:45.627930 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8ae457ca-61be-47ce-88d7-a2a348c0dd5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:45.757435 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae457ca-61be-47ce-88d7-a2a348c0dd5c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:45.765215 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4eb445-c848-4cb2-8e1d-f3e6ec65d3ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:45.765854 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4eb445-c848-4cb2-8e1d-f3e6ec65d3ef service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:45.766142 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4eb445-c848-4cb2-8e1d-f3e6ec65d3ef service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:45.766422 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4eb445-c848-4cb2-8e1d-f3e6ec65d3ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:45.849232 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4eb445-c848-4cb2-8e1d-f3e6ec65d3ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:45.851206 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4eb445-c848-4cb2-8e1d-f3e6ec65d3ef service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:45.866100 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1017/2040] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:53:45 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:53:52.717816 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e840f4-0c23-467e-8bd9-e83346e7f7d1 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:52.718349 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e840f4-0c23-467e-8bd9-e83346e7f7d1 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:52.718948 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e840f4-0c23-467e-8bd9-e83346e7f7d1 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:52.721220 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1024/2041] 10.222.0.97 () {54 vars in 921 bytes} [Tue Jul 26 14:53:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:53:52.729483 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc69f3c-2850-4bcc-a3c5-cf281319379a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:52.729979 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc69f3c-2850-4bcc-a3c5-cf281319379a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:52.730086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc69f3c-2850-4bcc-a3c5-cf281319379a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:52.757235 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-5cc69f3c-2850-4bcc-a3c5-cf281319379a None None] MFA Rules not processed for user `252c2d2939134416a3e8efb254f03c5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:52.758649 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5cc69f3c-2850-4bcc-a3c5-cf281319379a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:52.853658 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1018/2042] 10.222.0.97 () {58 vars in 1018 bytes} [Tue Jul 26 14:53:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:53:52.874498 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e840f4-0c23-467e-8bd9-e83346e7f7d1 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:52.877787 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c6e840f4-0c23-467e-8bd9-e83346e7f7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:52.880735 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc69f3c-2850-4bcc-a3c5-cf281319379a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:52.890467 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5cc69f3c-2850-4bcc-a3c5-cf281319379a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:52.907282 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:52.913369 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc69f3c-2850-4bcc-a3c5-cf281319379a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:52.918625 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5871b5-c0c8-4dc3-8fa0-3a2b25a2f9e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '252c2d2939134416a3e8efb254f03c5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:52.923210 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5871b5-c0c8-4dc3-8fa0-3a2b25a2f9e2 service placement] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:52.923210 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5871b5-c0c8-4dc3-8fa0-3a2b25a2f9e2 service placement] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:52.923210 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5871b5-c0c8-4dc3-8fa0-3a2b25a2f9e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:52.929046 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5871b5-c0c8-4dc3-8fa0-3a2b25a2f9e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:52.930670 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5871b5-c0c8-4dc3-8fa0-3a2b25a2f9e2 service placement] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:52.952695 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1019/2043] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:53:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:53:52.968948 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e840f4-0c23-467e-8bd9-e83346e7f7d1 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:52.974307 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4989834-4456-407d-bf2c-75f721dec405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:52.974960 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4989834-4456-407d-bf2c-75f721dec405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:52.975242 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4989834-4456-407d-bf2c-75f721dec405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:52.975587 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4989834-4456-407d-bf2c-75f721dec405 admin admin] PATH_INFO: `/v3/users/713d2a0f610d4296887b6ab9664ae5e6` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:52.976750 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4989834-4456-407d-bf2c-75f721dec405 admin admin] RBAC: Authorizing `identity:delete_user(user_id=713d2a0f610d4296887b6ab9664ae5e6)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:52.979011 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4989834-4456-407d-bf2c-75f721dec405 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:53.217466 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b4989834-4456-407d-bf2c-75f721dec405 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '713d2a0f610d4296887b6ab9664ae5e6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:53.218487 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b4989834-4456-407d-bf2c-75f721dec405 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '713d2a0f610d4296887b6ab9664ae5e6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:53.228263 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:53.234030 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b4989834-4456-407d-bf2c-75f721dec405 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '713d2a0f610d4296887b6ab9664ae5e6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:53.242270 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b4989834-4456-407d-bf2c-75f721dec405 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '713d2a0f610d4296887b6ab9664ae5e6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:53.243227 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b4989834-4456-407d-bf2c-75f721dec405 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '713d2a0f610d4296887b6ab9664ae5e6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:53.360545 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1025/2044] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:53:52 2022] DELETE /identity/v3/users/713d2a0f610d4296887b6ab9664ae5e6 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:53:53.662325 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5871b5-c0c8-4dc3-8fa0-3a2b25a2f9e2 service placement] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:53.678853 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5871b5-c0c8-4dc3-8fa0-3a2b25a2f9e2 service placement] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:53.738427 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e7fbe6-3266-49f6-884e-6cea0ecb1f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:53.739191 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7fbe6-3266-49f6-884e-6cea0ecb1f47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:53.739511 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7fbe6-3266-49f6-884e-6cea0ecb1f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:53.739643 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7fbe6-3266-49f6-884e-6cea0ecb1f47 admin admin] PATH_INFO: `/v3/projects/ca1bc66d9e5446758a759063544ae3e7` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:53.740800 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e7fbe6-3266-49f6-884e-6cea0ecb1f47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca1bc66d9e5446758a759063544ae3e7)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:53.743119 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e7fbe6-3266-49f6-884e-6cea0ecb1f47 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:53.914741 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-40e7fbe6-3266-49f6-884e-6cea0ecb1f47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca1bc66d9e5446758a759063544ae3e7', 'request_id': 'req-40e7fbe6-3266-49f6-884e-6cea0ecb1f47'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:53.920439 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-40e7fbe6-3266-49f6-884e-6cea0ecb1f47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca1bc66d9e5446758a759063544ae3e7', 'request_id': 'req-40e7fbe6-3266-49f6-884e-6cea0ecb1f47'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:53.994472 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-40e7fbe6-3266-49f6-884e-6cea0ecb1f47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca1bc66d9e5446758a759063544ae3e7', 'request_id': 'req-40e7fbe6-3266-49f6-884e-6cea0ecb1f47'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:54.013069 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-40e7fbe6-3266-49f6-884e-6cea0ecb1f47 admin admin] The token cache is being invalidate because project ca1bc66d9e5446758a759063544ae3e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:53:54.013365 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-40e7fbe6-3266-49f6-884e-6cea0ecb1f47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:54.014606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-40e7fbe6-3266-49f6-884e-6cea0ecb1f47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:54.016436 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1020/2045] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:53:53 2022] DELETE /identity/v3/projects/ca1bc66d9e5446758a759063544ae3e7 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:53:54.021623 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928d389-4e7a-466d-954f-5efee5309333 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:54.022102 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928d389-4e7a-466d-954f-5efee5309333 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:54.022224 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928d389-4e7a-466d-954f-5efee5309333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:54.042127 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-d928d389-4e7a-466d-954f-5efee5309333 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:54.043426 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d928d389-4e7a-466d-954f-5efee5309333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:54.111270 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1026/2046] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:53:54.124950 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8200e32-d628-4007-aef3-3a5224564bd9 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:54.124950 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8200e32-d628-4007-aef3-3a5224564bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:54.124950 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8200e32-d628-4007-aef3-3a5224564bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:54.146728 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-b8200e32-d628-4007-aef3-3a5224564bd9 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:54.148104 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b8200e32-d628-4007-aef3-3a5224564bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:54.156287 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1021/2047] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:53:54.161223 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d928d389-4e7a-466d-954f-5efee5309333 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:54.172217 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d928d389-4e7a-466d-954f-5efee5309333 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:54.178114 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0739fde-7163-427c-89dc-94d72147ac0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:54.178880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0739fde-7163-427c-89dc-94d72147ac0d admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:54.179162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0739fde-7163-427c-89dc-94d72147ac0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:54.179477 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0739fde-7163-427c-89dc-94d72147ac0d admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:54.180809 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0739fde-7163-427c-89dc-94d72147ac0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:53:54.185649 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0739fde-7163-427c-89dc-94d72147ac0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:54.185649 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0739fde-7163-427c-89dc-94d72147ac0d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:54.200759 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1027/2048] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:53:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:53:54.205644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8200e32-d628-4007-aef3-3a5224564bd9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:54.217065 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8200e32-d628-4007-aef3-3a5224564bd9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:54.222392 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d06f0-68e8-4d40-b776-7913985be7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:54.223202 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d06f0-68e8-4d40-b776-7913985be7ee admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:54.223642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d06f0-68e8-4d40-b776-7913985be7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:54.224133 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d06f0-68e8-4d40-b776-7913985be7ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:54.224953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d06f0-68e8-4d40-b776-7913985be7ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:54.227085 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d06f0-68e8-4d40-b776-7913985be7ee admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:54.355759 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1022/2049] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:53:54 2022] POST /identity/v3/projects => generated 365 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:53:54.360568 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0739fde-7163-427c-89dc-94d72147ac0d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:54.374000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0739fde-7163-427c-89dc-94d72147ac0d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:54.434823 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0473a-a9f7-4684-abc3-e083b5bdc0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:54.435504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0473a-a9f7-4684-abc3-e083b5bdc0af admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:54.435870 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0473a-a9f7-4684-abc3-e083b5bdc0af admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:54.436194 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0473a-a9f7-4684-abc3-e083b5bdc0af admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:54.437072 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c0473a-a9f7-4684-abc3-e083b5bdc0af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:54.439375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c0473a-a9f7-4684-abc3-e083b5bdc0af admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:54.554566 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1028/2050] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:53:54 2022] POST /identity/v3/users => generated 283 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:53:54.560047 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d06f0-68e8-4d40-b776-7913985be7ee admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:54.572951 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d06f0-68e8-4d40-b776-7913985be7ee admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:54.578161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1910ea-dc39-4b0b-9c26-da5c3b0baf52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:54.578795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1910ea-dc39-4b0b-9c26-da5c3b0baf52 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:54.579121 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1910ea-dc39-4b0b-9c26-da5c3b0baf52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:54.579339 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1910ea-dc39-4b0b-9c26-da5c3b0baf52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:54.580080 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1910ea-dc39-4b0b-9c26-da5c3b0baf52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:54.581678 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1910ea-dc39-4b0b-9c26-da5c3b0baf52 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:54.593497 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1023/2051] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:53:54.597631 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0473a-a9f7-4684-abc3-e083b5bdc0af admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:54.614796 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0473a-a9f7-4684-abc3-e083b5bdc0af admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:54.621147 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b539cfe-7085-444e-a0a7-5f2d9b94d20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:54.622108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b539cfe-7085-444e-a0a7-5f2d9b94d20c admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:54.622487 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b539cfe-7085-444e-a0a7-5f2d9b94d20c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:54.623739 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b539cfe-7085-444e-a0a7-5f2d9b94d20c admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:54.624641 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b539cfe-7085-444e-a0a7-5f2d9b94d20c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:54.626327 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b539cfe-7085-444e-a0a7-5f2d9b94d20c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:54.638550 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1029/2052] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:53:54.642983 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1910ea-dc39-4b0b-9c26-da5c3b0baf52 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:54.653230 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1910ea-dc39-4b0b-9c26-da5c3b0baf52 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:54.658458 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73a3f6-f1a2-4d81-a551-8029d73a8518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:54.659143 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73a3f6-f1a2-4d81-a551-8029d73a8518 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:54.659452 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73a3f6-f1a2-4d81-a551-8029d73a8518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:54.659734 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73a3f6-f1a2-4d81-a551-8029d73a8518 admin admin] PATH_INFO: `/v3/projects/d203b76aa92244d48e32dbc9e43b3cc8/users/645318a5f0164a32ab559213aa67462d/roles/e033961a9363456a8510c34414dd839e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:54.681815 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73a3f6-f1a2-4d81-a551-8029d73a8518 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d203b76aa92244d48e32dbc9e43b3cc8, user_id=645318a5f0164a32ab559213aa67462d, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:54.687023 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73a3f6-f1a2-4d81-a551-8029d73a8518 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:54.805132 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1024/2053] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:53:54 2022] PUT /identity/v3/projects/d203b76aa92244d48e32dbc9e43b3cc8/users/645318a5f0164a32ab559213aa67462d/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:53:54.817875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41399767-f29d-471f-a34d-8389e44b84a1 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:54.818174 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41399767-f29d-471f-a34d-8389e44b84a1 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:54.818449 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41399767-f29d-471f-a34d-8389e44b84a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:54.863658 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-41399767-f29d-471f-a34d-8389e44b84a1 None None] MFA Rules not processed for user `645318a5f0164a32ab559213aa67462d`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:54.865151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-41399767-f29d-471f-a34d-8389e44b84a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:54.916192 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:54.967951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1030/2054] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:54 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:53:54.972522 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73a3f6-f1a2-4d81-a551-8029d73a8518 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:54.991641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:55.039545 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73a3f6-f1a2-4d81-a551-8029d73a8518 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:55.046046 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701505c-10c9-4379-bac2-bd9355a8f83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:55.046677 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701505c-10c9-4379-bac2-bd9355a8f83c admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:55.046960 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701505c-10c9-4379-bac2-bd9355a8f83c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:55.047234 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701505c-10c9-4379-bac2-bd9355a8f83c admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:55.047959 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701505c-10c9-4379-bac2-bd9355a8f83c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:55.049893 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701505c-10c9-4379-bac2-bd9355a8f83c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:55.156264 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1025/2055] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:53:54 2022] POST /identity/v3/projects => generated 363 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:53:55.160575 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41399767-f29d-471f-a34d-8389e44b84a1 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:55.171594 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41399767-f29d-471f-a34d-8389e44b84a1 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:55.231297 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b7a5d-c5fb-41c8-87b6-f0ad22d66e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:55.231878 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b7a5d-c5fb-41c8-87b6-f0ad22d66e35 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:55.232161 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b7a5d-c5fb-41c8-87b6-f0ad22d66e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:55.232440 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b7a5d-c5fb-41c8-87b6-f0ad22d66e35 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:55.233164 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b7a5d-c5fb-41c8-87b6-f0ad22d66e35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:55.235603 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b7a5d-c5fb-41c8-87b6-f0ad22d66e35 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:55.320902 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1031/2056] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:53:55 2022] POST /identity/v3/users => generated 288 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:53:55.325479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701505c-10c9-4379-bac2-bd9355a8f83c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:55.337903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701505c-10c9-4379-bac2-bd9355a8f83c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:55.344212 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6ee3e-4574-473d-9c3a-51e3e47f4bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:55.344921 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6ee3e-4574-473d-9c3a-51e3e47f4bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:55.345289 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6ee3e-4574-473d-9c3a-51e3e47f4bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:55.345638 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6ee3e-4574-473d-9c3a-51e3e47f4bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:55.346413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f6ee3e-4574-473d-9c3a-51e3e47f4bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:55.348133 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f6ee3e-4574-473d-9c3a-51e3e47f4bc6 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:55.363319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1026/2057] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:53:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:53:55.364902 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b7a5d-c5fb-41c8-87b6-f0ad22d66e35 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:55.375347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b7a5d-c5fb-41c8-87b6-f0ad22d66e35 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:55.380863 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43dfaf3-0ba8-4c5d-af79-090ed655d53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:55.381545 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43dfaf3-0ba8-4c5d-af79-090ed655d53a admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:55.381856 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43dfaf3-0ba8-4c5d-af79-090ed655d53a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:55.382135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43dfaf3-0ba8-4c5d-af79-090ed655d53a admin admin] PATH_INFO: `/v3/projects/8040bd92227543feac958a865ec46780/users/a408f63dca4f4392bb176a50c0d72248/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:55.401289 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43dfaf3-0ba8-4c5d-af79-090ed655d53a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8040bd92227543feac958a865ec46780, user_id=a408f63dca4f4392bb176a50c0d72248, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:55.405162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43dfaf3-0ba8-4c5d-af79-090ed655d53a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:55.671428 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1032/2058] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:53:55 2022] PUT /identity/v3/projects/8040bd92227543feac958a865ec46780/users/a408f63dca4f4392bb176a50c0d72248/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:53:55.677522 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6ee3e-4574-473d-9c3a-51e3e47f4bc6 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:55.691530 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6ee3e-4574-473d-9c3a-51e3e47f4bc6 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:55.776020 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e995201-b0a8-401d-ad87-e7b5cedab77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:55.777012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e995201-b0a8-401d-ad87-e7b5cedab77b admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:55.777485 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e995201-b0a8-401d-ad87-e7b5cedab77b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:55.777957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e995201-b0a8-401d-ad87-e7b5cedab77b admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:55.779174 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e995201-b0a8-401d-ad87-e7b5cedab77b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:55.782026 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e995201-b0a8-401d-ad87-e7b5cedab77b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:55.802518 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1027/2059] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:53:55 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:53:55.805227 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43dfaf3-0ba8-4c5d-af79-090ed655d53a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:55.818081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43dfaf3-0ba8-4c5d-af79-090ed655d53a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:55.822747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb2322-8dbe-4cf6-9e29-eadc947ab49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:55.823448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb2322-8dbe-4cf6-9e29-eadc947ab49e admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:55.823741 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb2322-8dbe-4cf6-9e29-eadc947ab49e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:55.824022 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb2322-8dbe-4cf6-9e29-eadc947ab49e admin admin] PATH_INFO: `/v3/projects/8040bd92227543feac958a865ec46780/users/a408f63dca4f4392bb176a50c0d72248/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:55.826052 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb2322-8dbe-4cf6-9e29-eadc947ab49e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8040bd92227543feac958a865ec46780, user_id=a408f63dca4f4392bb176a50c0d72248, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:55.828770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb2322-8dbe-4cf6-9e29-eadc947ab49e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:55.839740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1033/2060] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:53:55 2022] PUT /identity/v3/projects/8040bd92227543feac958a865ec46780/users/a408f63dca4f4392bb176a50c0d72248/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:53:55.850375 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2cdc6-06b5-4bd0-a046-3da8d16eca77 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:55.851226 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2cdc6-06b5-4bd0-a046-3da8d16eca77 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:55.851226 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2cdc6-06b5-4bd0-a046-3da8d16eca77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:55.885969 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0cf2cdc6-06b5-4bd0-a046-3da8d16eca77 None None] MFA Rules not processed for user `a408f63dca4f4392bb176a50c0d72248`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:55.886784 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0cf2cdc6-06b5-4bd0-a046-3da8d16eca77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:56.014296 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1028/2061] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:55 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:53:56.070381 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb2322-8dbe-4cf6-9e29-eadc947ab49e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:56.073130 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b3cb2322-8dbe-4cf6-9e29-eadc947ab49e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:56.153154 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb2322-8dbe-4cf6-9e29-eadc947ab49e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:56.157417 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec3c02-481a-4f69-bd71-2bdf2c220546 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:56.158111 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec3c02-481a-4f69-bd71-2bdf2c220546 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:56.158407 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec3c02-481a-4f69-bd71-2bdf2c220546 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:56.158672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec3c02-481a-4f69-bd71-2bdf2c220546 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:56.170822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec3c02-481a-4f69-bd71-2bdf2c220546 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:56.172678 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec3c02-481a-4f69-bd71-2bdf2c220546 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:56.187471 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1034/2062] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:53:56 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:53:56.620130 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2cdc6-06b5-4bd0-a046-3da8d16eca77 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:56.640559 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2cdc6-06b5-4bd0-a046-3da8d16eca77 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:56.652750 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65af35-a4cc-40f4-a582-5838214161e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:56.653526 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65af35-a4cc-40f4-a582-5838214161e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:56.654108 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65af35-a4cc-40f4-a582-5838214161e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:56.654474 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65af35-a4cc-40f4-a582-5838214161e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:56.670689 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:56.727734 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65af35-a4cc-40f4-a582-5838214161e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:56.729313 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65af35-a4cc-40f4-a582-5838214161e2 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:56.739679 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1029/2063] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:53:56 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:53:57.152914 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec3c02-481a-4f69-bd71-2bdf2c220546 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:57.155616 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e9ec3c02-481a-4f69-bd71-2bdf2c220546 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:57.217645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:57.267612 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec3c02-481a-4f69-bd71-2bdf2c220546 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:57.292606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a772df8-9100-42f6-8a29-fe1530484dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:57.292606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a772df8-9100-42f6-8a29-fe1530484dd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:57.292606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a772df8-9100-42f6-8a29-fe1530484dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:57.292606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a772df8-9100-42f6-8a29-fe1530484dd1 admin admin] PATH_INFO: `/v3/users/eafdad5573674d99ab0010550c948e8b` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:57.296387 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a772df8-9100-42f6-8a29-fe1530484dd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eafdad5573674d99ab0010550c948e8b)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:57.300213 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a772df8-9100-42f6-8a29-fe1530484dd1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:57.658300 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0a772df8-9100-42f6-8a29-fe1530484dd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eafdad5573674d99ab0010550c948e8b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:57.659087 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0a772df8-9100-42f6-8a29-fe1530484dd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eafdad5573674d99ab0010550c948e8b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:57.672818 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0a772df8-9100-42f6-8a29-fe1530484dd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eafdad5573674d99ab0010550c948e8b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:57.683468 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0a772df8-9100-42f6-8a29-fe1530484dd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eafdad5573674d99ab0010550c948e8b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:57.684875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0a772df8-9100-42f6-8a29-fe1530484dd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eafdad5573674d99ab0010550c948e8b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:57.835042 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1035/2064] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:53:57 2022] DELETE /identity/v3/users/eafdad5573674d99ab0010550c948e8b => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:53:57.839950 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844854ba-410a-4530-8591-ee597cb0fa19 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:57.840762 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844854ba-410a-4530-8591-ee597cb0fa19 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:57.840929 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844854ba-410a-4530-8591-ee597cb0fa19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:57.864149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-844854ba-410a-4530-8591-ee597cb0fa19 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:57.865383 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-844854ba-410a-4530-8591-ee597cb0fa19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:57.933436 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1030/2065] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:53:57.944015 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a772df8-9100-42f6-8a29-fe1530484dd1 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:57.945940 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0a772df8-9100-42f6-8a29-fe1530484dd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:58.020899 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a772df8-9100-42f6-8a29-fe1530484dd1 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:58.026645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7842e24-2309-46c2-8fea-d14781dbfd5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:58.026645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7842e24-2309-46c2-8fea-d14781dbfd5a service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:58.026645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7842e24-2309-46c2-8fea-d14781dbfd5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:58.026645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7842e24-2309-46c2-8fea-d14781dbfd5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:58.038866 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7842e24-2309-46c2-8fea-d14781dbfd5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:58.041951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7842e24-2309-46c2-8fea-d14781dbfd5a service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:58.063671 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1036/2066] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:53:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:53:58.142031 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9dfee-50cb-4a76-8a9f-ecb24968ca89 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:58.142466 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9dfee-50cb-4a76-8a9f-ecb24968ca89 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:58.142919 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9dfee-50cb-4a76-8a9f-ecb24968ca89 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:58.145262 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1031/2067] 10.222.0.97 () {54 vars in 921 bytes} [Tue Jul 26 14:53:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:53:58.171889 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ddc087-7bea-4cf1-9a97-c1595b25ff07 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:58.172160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ddc087-7bea-4cf1-9a97-c1595b25ff07 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:58.172542 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ddc087-7bea-4cf1-9a97-c1595b25ff07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:58.193352 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-42ddc087-7bea-4cf1-9a97-c1595b25ff07 None None] MFA Rules not processed for user `252c2d2939134416a3e8efb254f03c5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:58.194471 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-42ddc087-7bea-4cf1-9a97-c1595b25ff07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:58.242157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:58.267315 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1037/2068] 10.222.0.97 () {58 vars in 1018 bytes} [Tue Jul 26 14:53:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:53:58.281638 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9dfee-50cb-4a76-8a9f-ecb24968ca89 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:58.283742 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-eac9dfee-50cb-4a76-8a9f-ecb24968ca89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:58.320751 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9dfee-50cb-4a76-8a9f-ecb24968ca89 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:58.345513 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ddc087-7bea-4cf1-9a97-c1595b25ff07 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:58.346806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8b7a6-4ddc-4aa0-96ca-89b2f683d669 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '252c2d2939134416a3e8efb254f03c5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:58.347104 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-42ddc087-7bea-4cf1-9a97-c1595b25ff07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:58.348812 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8b7a6-4ddc-4aa0-96ca-89b2f683d669 service placement] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:58.349452 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8b7a6-4ddc-4aa0-96ca-89b2f683d669 service placement] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:58.350256 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8b7a6-4ddc-4aa0-96ca-89b2f683d669 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:58.382165 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe8b7a6-4ddc-4aa0-96ca-89b2f683d669 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:58.382165 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe8b7a6-4ddc-4aa0-96ca-89b2f683d669 service placement] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:58.397095 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ddc087-7bea-4cf1-9a97-c1595b25ff07 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:58.409152 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1032/2069] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:53:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:53:58.416942 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde7e98b-2125-47f5-90a5-656989c79cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:58.418183 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7e98b-2125-47f5-90a5-656989c79cf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:58.418663 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7e98b-2125-47f5-90a5-656989c79cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:58.419138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7e98b-2125-47f5-90a5-656989c79cf6 admin admin] PATH_INFO: `/v3/users/222f461ac70d4423baf93df9109c7f08` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:58.421770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde7e98b-2125-47f5-90a5-656989c79cf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=222f461ac70d4423baf93df9109c7f08)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:58.429779 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde7e98b-2125-47f5-90a5-656989c79cf6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:58.701895 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8b7a6-4ddc-4aa0-96ca-89b2f683d669 service placement] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:58.704314 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1fe8b7a6-4ddc-4aa0-96ca-89b2f683d669 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:58.706313 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fde7e98b-2125-47f5-90a5-656989c79cf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '222f461ac70d4423baf93df9109c7f08'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:58.707176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fde7e98b-2125-47f5-90a5-656989c79cf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '222f461ac70d4423baf93df9109c7f08'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:58.728384 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fde7e98b-2125-47f5-90a5-656989c79cf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '222f461ac70d4423baf93df9109c7f08'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:58.738345 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fde7e98b-2125-47f5-90a5-656989c79cf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '222f461ac70d4423baf93df9109c7f08'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:58.739166 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fde7e98b-2125-47f5-90a5-656989c79cf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '222f461ac70d4423baf93df9109c7f08'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:58.739503 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8b7a6-4ddc-4aa0-96ca-89b2f683d669 service placement] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:58.749634 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa270e-7fb2-42a8-bdf2-66df58fb96e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:58.762164 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa270e-7fb2-42a8-bdf2-66df58fb96e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:58.762555 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa270e-7fb2-42a8-bdf2-66df58fb96e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:58.762928 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa270e-7fb2-42a8-bdf2-66df58fb96e1 admin admin] PATH_INFO: `/v3/projects/a4897a6c07e8482d9bbf88654e04f392` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:58.764983 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aa270e-7fb2-42a8-bdf2-66df58fb96e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4897a6c07e8482d9bbf88654e04f392)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:58.767676 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aa270e-7fb2-42a8-bdf2-66df58fb96e1 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:58.830036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1038/2070] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:53:58 2022] DELETE /identity/v3/users/222f461ac70d4423baf93df9109c7f08 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:53:58.926042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-10aa270e-7fb2-42a8-bdf2-66df58fb96e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4897a6c07e8482d9bbf88654e04f392', 'request_id': 'req-10aa270e-7fb2-42a8-bdf2-66df58fb96e1'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:58.932474 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-10aa270e-7fb2-42a8-bdf2-66df58fb96e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4897a6c07e8482d9bbf88654e04f392', 'request_id': 'req-10aa270e-7fb2-42a8-bdf2-66df58fb96e1'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:59.028987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-10aa270e-7fb2-42a8-bdf2-66df58fb96e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4897a6c07e8482d9bbf88654e04f392', 'request_id': 'req-10aa270e-7fb2-42a8-bdf2-66df58fb96e1'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:59.041834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-10aa270e-7fb2-42a8-bdf2-66df58fb96e1 admin admin] The token cache is being invalidate because project a4897a6c07e8482d9bbf88654e04f392 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:53:59.042254 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-10aa270e-7fb2-42a8-bdf2-66df58fb96e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:59.042942 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-10aa270e-7fb2-42a8-bdf2-66df58fb96e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:59.045292 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1033/2071] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:53:58 2022] DELETE /identity/v3/projects/a4897a6c07e8482d9bbf88654e04f392 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:53:59.050544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2236d3e-c26c-428a-84d0-4bb44e5b1571 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:59.050862 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2236d3e-c26c-428a-84d0-4bb44e5b1571 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:59.051145 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2236d3e-c26c-428a-84d0-4bb44e5b1571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:59.072335 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-d2236d3e-c26c-428a-84d0-4bb44e5b1571 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:59.073598 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d2236d3e-c26c-428a-84d0-4bb44e5b1571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:59.140145 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1039/2072] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:53:59.148223 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1609c-69ae-4011-a6c6-d369a02f3269 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:59.148573 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1609c-69ae-4011-a6c6-d369a02f3269 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:59.148913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1609c-69ae-4011-a6c6-d369a02f3269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:59.152363 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2236d3e-c26c-428a-84d0-4bb44e5b1571 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:59.155134 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d2236d3e-c26c-428a-84d0-4bb44e5b1571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:59.169552 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2236d3e-c26c-428a-84d0-4bb44e5b1571 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:59.172256 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-4ae1609c-69ae-4011-a6c6-d369a02f3269 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:53:59.173567 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4ae1609c-69ae-4011-a6c6-d369a02f3269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:59.181257 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4919b-66ca-4d7b-bc0c-da8f767d83a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:59.182306 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4919b-66ca-4d7b-bc0c-da8f767d83a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:59.187271 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4919b-66ca-4d7b-bc0c-da8f767d83a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:59.187691 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1034/2073] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:53:59.187884 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4919b-66ca-4d7b-bc0c-da8f767d83a6 admin admin] PATH_INFO: `/v3/projects/ffe2a45945754ec7a7dac33c7eb5077c` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:59.187884 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee4919b-66ca-4d7b-bc0c-da8f767d83a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffe2a45945754ec7a7dac33c7eb5077c)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:59.190221 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae1609c-69ae-4011-a6c6-d369a02f3269 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:59.190919 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee4919b-66ca-4d7b-bc0c-da8f767d83a6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:59.200580 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae1609c-69ae-4011-a6c6-d369a02f3269 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:59.206164 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d875bc-6c93-4457-8643-42eda1864856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:59.206791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d875bc-6c93-4457-8643-42eda1864856 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:59.207072 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d875bc-6c93-4457-8643-42eda1864856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:59.207350 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d875bc-6c93-4457-8643-42eda1864856 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:59.208147 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d875bc-6c93-4457-8643-42eda1864856 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:53:59.208397 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d875bc-6c93-4457-8643-42eda1864856 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:59.210060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d875bc-6c93-4457-8643-42eda1864856 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:59.219864 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:59.223654 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1035/2074] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:53:59.234614 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d875bc-6c93-4457-8643-42eda1864856 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:59.244839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d875bc-6c93-4457-8643-42eda1864856 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:59.250402 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a27fc1-00f8-4b92-83c8-bed2a2fa5833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:59.251035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a27fc1-00f8-4b92-83c8-bed2a2fa5833 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:59.251319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a27fc1-00f8-4b92-83c8-bed2a2fa5833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:59.251627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a27fc1-00f8-4b92-83c8-bed2a2fa5833 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:59.252306 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a27fc1-00f8-4b92-83c8-bed2a2fa5833 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:59.254226 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a27fc1-00f8-4b92-83c8-bed2a2fa5833 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:59.354071 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9ee4919b-66ca-4d7b-bc0c-da8f767d83a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffe2a45945754ec7a7dac33c7eb5077c', 'request_id': 'req-9ee4919b-66ca-4d7b-bc0c-da8f767d83a6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:59.358776 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:53:59.360908 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9ee4919b-66ca-4d7b-bc0c-da8f767d83a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffe2a45945754ec7a7dac33c7eb5077c', 'request_id': 'req-9ee4919b-66ca-4d7b-bc0c-da8f767d83a6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:59.418838 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1036/2075] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:53:59 2022] POST /identity/v3/projects => generated 383 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:53:59.421006 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a27fc1-00f8-4b92-83c8-bed2a2fa5833 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:59.468980 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9ee4919b-66ca-4d7b-bc0c-da8f767d83a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffe2a45945754ec7a7dac33c7eb5077c', 'request_id': 'req-9ee4919b-66ca-4d7b-bc0c-da8f767d83a6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:59.489926 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9ee4919b-66ca-4d7b-bc0c-da8f767d83a6 admin admin] The token cache is being invalidate because project ffe2a45945754ec7a7dac33c7eb5077c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:53:59.490604 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9ee4919b-66ca-4d7b-bc0c-da8f767d83a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:59.491660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9ee4919b-66ca-4d7b-bc0c-da8f767d83a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:59.494535 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1040/2076] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:53:59 2022] DELETE /identity/v3/projects/ffe2a45945754ec7a7dac33c7eb5077c => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:53:59.500644 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4919b-66ca-4d7b-bc0c-da8f767d83a6 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:59.502851 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9ee4919b-66ca-4d7b-bc0c-da8f767d83a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:59.503959 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a27fc1-00f8-4b92-83c8-bed2a2fa5833 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:59.511659 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b8c69-197f-466d-91ec-23ff0468227c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:59.512590 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b8c69-197f-466d-91ec-23ff0468227c admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:59.513036 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b8c69-197f-466d-91ec-23ff0468227c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:59.514044 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b8c69-197f-466d-91ec-23ff0468227c admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:59.515175 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428b8c69-197f-466d-91ec-23ff0468227c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:59.518041 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428b8c69-197f-466d-91ec-23ff0468227c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:59.587001 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4919b-66ca-4d7b-bc0c-da8f767d83a6 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:59.594280 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1906a27-903a-4c20-a449-1174a559ad5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:59.595622 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1906a27-903a-4c20-a449-1174a559ad5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:59.596124 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1906a27-903a-4c20-a449-1174a559ad5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:59.596582 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1906a27-903a-4c20-a449-1174a559ad5a admin admin] PATH_INFO: `/v3/users/edc66fda91e34a2083ceba934da3dcd6` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:59.598775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1906a27-903a-4c20-a449-1174a559ad5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=edc66fda91e34a2083ceba934da3dcd6)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:59.603928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1906a27-903a-4c20-a449-1174a559ad5a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:59.663936 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1037/2077] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:53:59 2022] POST /identity/v3/users => generated 292 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:53:59.668726 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b8c69-197f-466d-91ec-23ff0468227c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:59.670672 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-428b8c69-197f-466d-91ec-23ff0468227c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:53:59.682878 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b8c69-197f-466d-91ec-23ff0468227c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:59.687879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6ba3c-f507-469c-b606-7365bfd38aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:59.688506 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6ba3c-f507-469c-b606-7365bfd38aea admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:59.688789 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6ba3c-f507-469c-b606-7365bfd38aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:59.689066 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6ba3c-f507-469c-b606-7365bfd38aea admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:59.689880 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6ba3c-f507-469c-b606-7365bfd38aea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:59.696975 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6ba3c-f507-469c-b606-7365bfd38aea admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:59.707138 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1038/2078] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:53:59.710956 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6ba3c-f507-469c-b606-7365bfd38aea admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:59.721421 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6ba3c-f507-469c-b606-7365bfd38aea admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:59.726119 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c669b-2314-407c-9942-38fc466a6129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:59.726722 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c669b-2314-407c-9942-38fc466a6129 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:59.727006 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c669b-2314-407c-9942-38fc466a6129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:59.727280 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c669b-2314-407c-9942-38fc466a6129 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:59.728004 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59c669b-2314-407c-9942-38fc466a6129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:59.729764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59c669b-2314-407c-9942-38fc466a6129 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:59.741624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1039/2079] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:53:59.745754 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c669b-2314-407c-9942-38fc466a6129 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:53:59.759950 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c669b-2314-407c-9942-38fc466a6129 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:53:59.770045 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820f19fe-c1f7-43e2-a01f-0561e4683782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:53:59.771412 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f19fe-c1f7-43e2-a01f-0561e4683782 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:53:59.771987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f19fe-c1f7-43e2-a01f-0561e4683782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:53:59.772537 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f19fe-c1f7-43e2-a01f-0561e4683782 admin admin] PATH_INFO: `/v3/projects/5e97a678d3d1434387855e553c35e55a/users/aacb04366c3a4eca9aadd56ea31b7acb/roles/e033961a9363456a8510c34414dd839e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:53:59.796582 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820f19fe-c1f7-43e2-a01f-0561e4683782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e97a678d3d1434387855e553c35e55a, user_id=aacb04366c3a4eca9aadd56ea31b7acb, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:53:59.799339 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820f19fe-c1f7-43e2-a01f-0561e4683782 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:53:59.963797 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b1906a27-903a-4c20-a449-1174a559ad5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edc66fda91e34a2083ceba934da3dcd6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:59.965268 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b1906a27-903a-4c20-a449-1174a559ad5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edc66fda91e34a2083ceba934da3dcd6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:59.978731 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b1906a27-903a-4c20-a449-1174a559ad5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edc66fda91e34a2083ceba934da3dcd6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:59.987739 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b1906a27-903a-4c20-a449-1174a559ad5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edc66fda91e34a2083ceba934da3dcd6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:59.993411 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b1906a27-903a-4c20-a449-1174a559ad5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edc66fda91e34a2083ceba934da3dcd6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:53:59.993556 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1040/2080] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:53:59 2022] PUT /identity/v3/projects/5e97a678d3d1434387855e553c35e55a/users/aacb04366c3a4eca9aadd56ea31b7acb/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:54:00.004353 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa2399-d411-459b-9afd-68229d90c40a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:00.004738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa2399-d411-459b-9afd-68229d90c40a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:00.005110 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa2399-d411-459b-9afd-68229d90c40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:00.034702 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f3aa2399-d411-459b-9afd-68229d90c40a None None] MFA Rules not processed for user `aacb04366c3a4eca9aadd56ea31b7acb`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:00.035954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f3aa2399-d411-459b-9afd-68229d90c40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:00.102419 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1041/2081] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:54:00.106827 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa2399-d411-459b-9afd-68229d90c40a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:00.108808 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f3aa2399-d411-459b-9afd-68229d90c40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:00.123285 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:00.174916 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa2399-d411-459b-9afd-68229d90c40a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:00.179484 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f614f98-cccf-4d30-a92c-808efdb9cc2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:00.180103 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f614f98-cccf-4d30-a92c-808efdb9cc2a admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:00.180387 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f614f98-cccf-4d30-a92c-808efdb9cc2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:00.180660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f614f98-cccf-4d30-a92c-808efdb9cc2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:00.181322 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f614f98-cccf-4d30-a92c-808efdb9cc2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:00.183228 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f614f98-cccf-4d30-a92c-808efdb9cc2a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:00.201390 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1041/2082] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:53:59 2022] DELETE /identity/v3/users/edc66fda91e34a2083ceba934da3dcd6 => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:54:00.335220 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1042/2083] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:54:00 2022] POST /identity/v3/projects => generated 385 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:54:00.339006 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1906a27-903a-4c20-a449-1174a559ad5a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:00.416339 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1906a27-903a-4c20-a449-1174a559ad5a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:00.423003 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:00.520177 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f98c2-f643-4e98-bd76-fbb30be3406c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:00.520177 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f98c2-f643-4e98-bd76-fbb30be3406c admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:00.520177 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f98c2-f643-4e98-bd76-fbb30be3406c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:00.520177 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f98c2-f643-4e98-bd76-fbb30be3406c admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:00.520694 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f98c2-f643-4e98-bd76-fbb30be3406c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:00.520694 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f98c2-f643-4e98-bd76-fbb30be3406c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:00.648908 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f614f98-cccf-4d30-a92c-808efdb9cc2a admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:00.652791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2f614f98-cccf-4d30-a92c-808efdb9cc2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:00.667349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f614f98-cccf-4d30-a92c-808efdb9cc2a admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:00.673708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaec53b-9396-4eb0-a6cb-d2f13d9d7ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:00.674566 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaec53b-9396-4eb0-a6cb-d2f13d9d7ac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:00.674957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaec53b-9396-4eb0-a6cb-d2f13d9d7ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:00.675329 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaec53b-9396-4eb0-a6cb-d2f13d9d7ac6 admin admin] PATH_INFO: `/v3/projects/e7c8bb84a38942fa9ba0680d0fb97031` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:00.676832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaec53b-9396-4eb0-a6cb-d2f13d9d7ac6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7c8bb84a38942fa9ba0680d0fb97031)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:00.678995 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaec53b-9396-4eb0-a6cb-d2f13d9d7ac6 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:01.014039 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1042/2084] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:54:00 2022] POST /identity/v3/users => generated 293 bytes in 674 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:54:01.017456 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f98c2-f643-4e98-bd76-fbb30be3406c admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:01.027341 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f98c2-f643-4e98-bd76-fbb30be3406c admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:01.033504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf8f45-15ca-406d-ad84-5ea8f0669ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:01.035712 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf8f45-15ca-406d-ad84-5ea8f0669ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:01.036174 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf8f45-15ca-406d-ad84-5ea8f0669ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:01.040476 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf8f45-15ca-406d-ad84-5ea8f0669ce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:01.045649 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaf8f45-15ca-406d-ad84-5ea8f0669ce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:01.047335 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaf8f45-15ca-406d-ad84-5ea8f0669ce9 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:01.065041 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1043/2085] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:54:01.066156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf8f45-15ca-406d-ad84-5ea8f0669ce9 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:01.076772 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf8f45-15ca-406d-ad84-5ea8f0669ce9 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:01.084802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2aaec53b-9396-4eb0-a6cb-d2f13d9d7ac6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7c8bb84a38942fa9ba0680d0fb97031', 'request_id': 'req-2aaec53b-9396-4eb0-a6cb-d2f13d9d7ac6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:01.095224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2aaec53b-9396-4eb0-a6cb-d2f13d9d7ac6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7c8bb84a38942fa9ba0680d0fb97031', 'request_id': 'req-2aaec53b-9396-4eb0-a6cb-d2f13d9d7ac6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:01.139596 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b6116-376a-4329-b3dd-cdf15b6b2d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:01.140224 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b6116-376a-4329-b3dd-cdf15b6b2d15 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:01.140611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b6116-376a-4329-b3dd-cdf15b6b2d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:01.140783 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b6116-376a-4329-b3dd-cdf15b6b2d15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:01.141452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b6116-376a-4329-b3dd-cdf15b6b2d15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:01.143055 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b6116-376a-4329-b3dd-cdf15b6b2d15 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:01.161483 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1044/2086] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:54:01.165885 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b6116-376a-4329-b3dd-cdf15b6b2d15 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:01.176566 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b6116-376a-4329-b3dd-cdf15b6b2d15 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:01.177439 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2aaec53b-9396-4eb0-a6cb-d2f13d9d7ac6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7c8bb84a38942fa9ba0680d0fb97031', 'request_id': 'req-2aaec53b-9396-4eb0-a6cb-d2f13d9d7ac6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:01.189962 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1c115-3830-4780-90d8-93c00c6e50df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:01.190641 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1c115-3830-4780-90d8-93c00c6e50df admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:01.190921 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1c115-3830-4780-90d8-93c00c6e50df admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:01.191198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1c115-3830-4780-90d8-93c00c6e50df admin admin] PATH_INFO: `/v3/projects/446ab7d8cce84609b469dff72678ed91/users/db3824759bb3467583616967d9e66db7/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:01.194530 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2aaec53b-9396-4eb0-a6cb-d2f13d9d7ac6 admin admin] The token cache is being invalidate because project e7c8bb84a38942fa9ba0680d0fb97031 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:54:01.194930 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2aaec53b-9396-4eb0-a6cb-d2f13d9d7ac6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:01.195743 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2aaec53b-9396-4eb0-a6cb-d2f13d9d7ac6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:01.197394 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1043/2087] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:54:00 2022] DELETE /identity/v3/projects/e7c8bb84a38942fa9ba0680d0fb97031 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:01.203160 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca4453-a312-4fc1-88f0-a37bc0017e58 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:01.203559 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca4453-a312-4fc1-88f0-a37bc0017e58 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:01.203984 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca4453-a312-4fc1-88f0-a37bc0017e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:01.207648 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1c115-3830-4780-90d8-93c00c6e50df admin admin] RBAC: Authorizing `identity:create_grant(project_id=446ab7d8cce84609b469dff72678ed91, user_id=db3824759bb3467583616967d9e66db7, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:01.210342 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1c115-3830-4780-90d8-93c00c6e50df admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:01.226013 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0dca4453-a312-4fc1-88f0-a37bc0017e58 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:01.227577 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0dca4453-a312-4fc1-88f0-a37bc0017e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:01.236909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1044/2088] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:54:01.244452 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62600d1-f9e6-49eb-b006-4ea2c55d4a1f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:01.245497 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62600d1-f9e6-49eb-b006-4ea2c55d4a1f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:01.245497 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62600d1-f9e6-49eb-b006-4ea2c55d4a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:01.265387 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f62600d1-f9e6-49eb-b006-4ea2c55d4a1f None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:01.266590 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f62600d1-f9e6-49eb-b006-4ea2c55d4a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:01.281139 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1045/2089] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:54:01.285507 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62600d1-f9e6-49eb-b006-4ea2c55d4a1f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:01.286928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1045/2090] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:54:01 2022] PUT /identity/v3/projects/446ab7d8cce84609b469dff72678ed91/users/db3824759bb3467583616967d9e66db7/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:54:01.295459 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62600d1-f9e6-49eb-b006-4ea2c55d4a1f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:01.299058 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c0dd2-534f-41e3-a72a-51d4ed3797bc None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:01.299812 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c0dd2-534f-41e3-a72a-51d4ed3797bc None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:01.300266 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c0dd2-534f-41e3-a72a-51d4ed3797bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:01.343728 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-1f7c0dd2-534f-41e3-a72a-51d4ed3797bc None None] MFA Rules not processed for user `db3824759bb3467583616967d9e66db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:01.345572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1f7c0dd2-534f-41e3-a72a-51d4ed3797bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:01.388982 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a201b23-8eaf-4741-adba-5b2113bd13a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:01.397713 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a201b23-8eaf-4741-adba-5b2113bd13a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:01.397713 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a201b23-8eaf-4741-adba-5b2113bd13a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:01.397713 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a201b23-8eaf-4741-adba-5b2113bd13a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:01.397713 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a201b23-8eaf-4741-adba-5b2113bd13a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:54:01.397713 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a201b23-8eaf-4741-adba-5b2113bd13a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:01.397713 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a201b23-8eaf-4741-adba-5b2113bd13a9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:01.413820 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1046/2091] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:54:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:54:01.417627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:01.423156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a201b23-8eaf-4741-adba-5b2113bd13a9 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:01.434829 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a201b23-8eaf-4741-adba-5b2113bd13a9 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:01.436365 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1046/2092] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:54:01.440845 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c0dd2-534f-41e3-a72a-51d4ed3797bc None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:01.441360 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ee257-eccb-47bd-b334-069eecd906cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:01.442306 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1f7c0dd2-534f-41e3-a72a-51d4ed3797bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:01.442415 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ee257-eccb-47bd-b334-069eecd906cc admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:01.442738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ee257-eccb-47bd-b334-069eecd906cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:01.443073 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ee257-eccb-47bd-b334-069eecd906cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:01.443843 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ee257-eccb-47bd-b334-069eecd906cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:01.446335 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ee257-eccb-47bd-b334-069eecd906cc admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:01.458483 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c0dd2-534f-41e3-a72a-51d4ed3797bc None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:01.467525 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b7142-1995-41d2-b463-171411d021de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:01.469071 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b7142-1995-41d2-b463-171411d021de admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:01.469166 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b7142-1995-41d2-b463-171411d021de admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:01.469826 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b7142-1995-41d2-b463-171411d021de admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:01.471258 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b7142-1995-41d2-b463-171411d021de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:01.478486 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b7142-1995-41d2-b463-171411d021de admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:01.545735 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1047/2093] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:54:01 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:54:01.549820 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ee257-eccb-47bd-b334-069eecd906cc admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:01.616265 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ee257-eccb-47bd-b334-069eecd906cc admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:01.621468 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027f086-e46e-4d5e-a76e-45b00917521e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:01.622089 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027f086-e46e-4d5e-a76e-45b00917521e admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:01.622373 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027f086-e46e-4d5e-a76e-45b00917521e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:01.622650 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027f086-e46e-4d5e-a76e-45b00917521e admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:01.623964 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6027f086-e46e-4d5e-a76e-45b00917521e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:01.626009 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6027f086-e46e-4d5e-a76e-45b00917521e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:01.653412 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1047/2094] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:54:01 2022] POST /identity/v3/projects => generated 385 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:54:01.658057 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b7142-1995-41d2-b463-171411d021de admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:01.728790 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b7142-1995-41d2-b463-171411d021de admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:01.733682 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077f403-dc30-478d-b1da-3eea852be232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:01.734311 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077f403-dc30-478d-b1da-3eea852be232 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:01.734595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077f403-dc30-478d-b1da-3eea852be232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:01.734877 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077f403-dc30-478d-b1da-3eea852be232 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:01.735576 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077f403-dc30-478d-b1da-3eea852be232 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:01.737577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077f403-dc30-478d-b1da-3eea852be232 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:01.951677 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1048/2095] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:54:01 2022] POST /identity/v3/users => generated 288 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:54:01.951677 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027f086-e46e-4d5e-a76e-45b00917521e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:01.951677 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027f086-e46e-4d5e-a76e-45b00917521e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:01.951677 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00d19f-0f72-4d75-ad5e-36f88c9f3bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:01.952315 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00d19f-0f72-4d75-ad5e-36f88c9f3bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:01.952315 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00d19f-0f72-4d75-ad5e-36f88c9f3bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:01.952315 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00d19f-0f72-4d75-ad5e-36f88c9f3bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:01.952315 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f00d19f-0f72-4d75-ad5e-36f88c9f3bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:01.952315 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f00d19f-0f72-4d75-ad5e-36f88c9f3bc6 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:01.960540 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1049/2096] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:54:01.964500 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00d19f-0f72-4d75-ad5e-36f88c9f3bc6 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:01.975203 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00d19f-0f72-4d75-ad5e-36f88c9f3bc6 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:01.979976 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac55e4-e1cd-4250-9955-43669ac9a5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:01.980582 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac55e4-e1cd-4250-9955-43669ac9a5de admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:01.980873 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac55e4-e1cd-4250-9955-43669ac9a5de admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:01.981148 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac55e4-e1cd-4250-9955-43669ac9a5de admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:01.981837 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac55e4-e1cd-4250-9955-43669ac9a5de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:01.983966 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac55e4-e1cd-4250-9955-43669ac9a5de admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:01.995034 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1050/2097] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:54:01.999351 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac55e4-e1cd-4250-9955-43669ac9a5de admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:02.009033 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac55e4-e1cd-4250-9955-43669ac9a5de admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:02.014030 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99ea1b-69af-4936-be73-e6d5ef36d937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:02.014681 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99ea1b-69af-4936-be73-e6d5ef36d937 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:02.014965 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99ea1b-69af-4936-be73-e6d5ef36d937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:02.015242 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99ea1b-69af-4936-be73-e6d5ef36d937 admin admin] PATH_INFO: `/v3/projects/79d3d651ad9f465fa235688c586d01cc/users/ad85ed13d8324a1d8ab49e14f92d0f6a/roles/e033961a9363456a8510c34414dd839e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:02.029929 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca99ea1b-69af-4936-be73-e6d5ef36d937 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79d3d651ad9f465fa235688c586d01cc, user_id=ad85ed13d8324a1d8ab49e14f92d0f6a, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:02.032683 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca99ea1b-69af-4936-be73-e6d5ef36d937 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:02.069199 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1048/2098] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:54:01 2022] POST /identity/v3/users => generated 299 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:54:02.073397 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077f403-dc30-478d-b1da-3eea852be232 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:02.083120 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077f403-dc30-478d-b1da-3eea852be232 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:02.087769 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffec7a-20b8-4b78-999e-085d8e39fba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:02.088480 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffec7a-20b8-4b78-999e-085d8e39fba3 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:02.088765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffec7a-20b8-4b78-999e-085d8e39fba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:02.089042 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffec7a-20b8-4b78-999e-085d8e39fba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:02.089864 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ffec7a-20b8-4b78-999e-085d8e39fba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:02.091971 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ffec7a-20b8-4b78-999e-085d8e39fba3 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:02.103226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1049/2099] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:54:02.108088 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffec7a-20b8-4b78-999e-085d8e39fba3 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:02.118392 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffec7a-20b8-4b78-999e-085d8e39fba3 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:02.123712 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ef1bb-18b3-456f-805e-a5a52ad9f752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:02.124372 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ef1bb-18b3-456f-805e-a5a52ad9f752 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:02.124662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ef1bb-18b3-456f-805e-a5a52ad9f752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:02.124942 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ef1bb-18b3-456f-805e-a5a52ad9f752 admin admin] PATH_INFO: `/v3/projects/2bf913b7df7e400e8b47f2fa67045763/users/0c7f376d95ec4d1db0079c448e7b24b7/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:02.140347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ef1bb-18b3-456f-805e-a5a52ad9f752 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bf913b7df7e400e8b47f2fa67045763, user_id=0c7f376d95ec4d1db0079c448e7b24b7, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:02.143014 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ef1bb-18b3-456f-805e-a5a52ad9f752 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:02.185844 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1051/2100] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:54:01 2022] PUT /identity/v3/projects/79d3d651ad9f465fa235688c586d01cc/users/ad85ed13d8324a1d8ab49e14f92d0f6a/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:54:02.189472 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:02.196714 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175057cd-84ac-4773-8ea3-6565f2ee81d2 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:02.197120 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175057cd-84ac-4773-8ea3-6565f2ee81d2 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:02.197492 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175057cd-84ac-4773-8ea3-6565f2ee81d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:02.230373 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-175057cd-84ac-4773-8ea3-6565f2ee81d2 None None] MFA Rules not processed for user `ad85ed13d8324a1d8ab49e14f92d0f6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:02.231542 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-175057cd-84ac-4773-8ea3-6565f2ee81d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:02.288883 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1050/2101] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:54:02 2022] PUT /identity/v3/projects/2bf913b7df7e400e8b47f2fa67045763/users/0c7f376d95ec4d1db0079c448e7b24b7/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:54:02.294320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ef1bb-18b3-456f-805e-a5a52ad9f752 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:02.312511 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1052/2102] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:54:02.317196 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175057cd-84ac-4773-8ea3-6565f2ee81d2 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:02.380568 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ef1bb-18b3-456f-805e-a5a52ad9f752 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:02.387028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e7ad7-a958-4819-b12f-ee480512fddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:02.388070 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e7ad7-a958-4819-b12f-ee480512fddf admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:02.388764 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e7ad7-a958-4819-b12f-ee480512fddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:02.389481 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e7ad7-a958-4819-b12f-ee480512fddf admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:02.390540 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801e7ad7-a958-4819-b12f-ee480512fddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:02.393793 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801e7ad7-a958-4819-b12f-ee480512fddf admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:02.396592 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175057cd-84ac-4773-8ea3-6565f2ee81d2 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:02.402436 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33205483-8e16-4501-94d9-f7351f530cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:02.403064 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33205483-8e16-4501-94d9-f7351f530cc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:02.405388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33205483-8e16-4501-94d9-f7351f530cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:02.405388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33205483-8e16-4501-94d9-f7351f530cc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:02.405388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33205483-8e16-4501-94d9-f7351f530cc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:02.406804 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33205483-8e16-4501-94d9-f7351f530cc2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:02.409201 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1051/2103] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:54:02.413135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e7ad7-a958-4819-b12f-ee480512fddf admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:02.423973 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e7ad7-a958-4819-b12f-ee480512fddf admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:02.426774 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:02.429173 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683efa70-f2ca-4bea-8bf0-062641f968e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:02.429833 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683efa70-f2ca-4bea-8bf0-062641f968e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:02.430117 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683efa70-f2ca-4bea-8bf0-062641f968e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:02.434389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683efa70-f2ca-4bea-8bf0-062641f968e4 admin admin] PATH_INFO: `/v3/projects/2bf913b7df7e400e8b47f2fa67045763/users/0c7f376d95ec4d1db0079c448e7b24b7/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:02.434389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683efa70-f2ca-4bea-8bf0-062641f968e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bf913b7df7e400e8b47f2fa67045763, user_id=0c7f376d95ec4d1db0079c448e7b24b7, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:02.436105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683efa70-f2ca-4bea-8bf0-062641f968e4 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:02.446597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1052/2104] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:54:02 2022] PUT /identity/v3/projects/2bf913b7df7e400e8b47f2fa67045763/users/0c7f376d95ec4d1db0079c448e7b24b7/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:54:02.458446 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13c462-e023-4753-8fc4-64018391e070 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:02.459817 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13c462-e023-4753-8fc4-64018391e070 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:02.459817 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13c462-e023-4753-8fc4-64018391e070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:02.466311 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:02.490689 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-5d13c462-e023-4753-8fc4-64018391e070 None None] MFA Rules not processed for user `0c7f376d95ec4d1db0079c448e7b24b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:02.492504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5d13c462-e023-4753-8fc4-64018391e070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:02.578689 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1053/2105] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:54:02.590176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13c462-e023-4753-8fc4-64018391e070 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:02.591763 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5d13c462-e023-4753-8fc4-64018391e070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:02.594581 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1053/2106] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:54:02 2022] POST /identity/v3/projects => generated 375 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:54:02.599171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33205483-8e16-4501-94d9-f7351f530cc2 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:02.656916 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13c462-e023-4753-8fc4-64018391e070 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:02.668634 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33205483-8e16-4501-94d9-f7351f530cc2 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:02.674023 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7caea30-a4b4-4923-bf83-073347ca29a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:02.674647 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7caea30-a4b4-4923-bf83-073347ca29a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:02.674931 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7caea30-a4b4-4923-bf83-073347ca29a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:02.675214 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7caea30-a4b4-4923-bf83-073347ca29a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:02.675939 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7caea30-a4b4-4923-bf83-073347ca29a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:02.677870 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7caea30-a4b4-4923-bf83-073347ca29a4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:02.712298 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4800e32-33a6-499c-bc39-2467ef3b2b78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:02.712981 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4800e32-33a6-499c-bc39-2467ef3b2b78 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:02.713300 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4800e32-33a6-499c-bc39-2467ef3b2b78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:02.713618 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4800e32-33a6-499c-bc39-2467ef3b2b78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:02.715715 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e4800e32-33a6-499c-bc39-2467ef3b2b78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:02.769033 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1054/2107] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:54:02 2022] POST /identity/v3/users => generated 294 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:54:02.770908 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7caea30-a4b4-4923-bf83-073347ca29a4 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:02.785454 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7caea30-a4b4-4923-bf83-073347ca29a4 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:02.792221 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65e5b6-e5d5-432d-9730-806336004151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:02.793337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65e5b6-e5d5-432d-9730-806336004151 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:02.793830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65e5b6-e5d5-432d-9730-806336004151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:02.794252 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65e5b6-e5d5-432d-9730-806336004151 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:02.795765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b65e5b6-e5d5-432d-9730-806336004151 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:02.799061 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b65e5b6-e5d5-432d-9730-806336004151 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:02.803883 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4800e32-33a6-499c-bc39-2467ef3b2b78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:02.806568 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4800e32-33a6-499c-bc39-2467ef3b2b78 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:02.815139 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:02.820400 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1054/2108] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:54:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:54:02.829179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1055/2109] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:54:02.832754 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4800e32-33a6-499c-bc39-2467ef3b2b78 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:02.843604 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4800e32-33a6-499c-bc39-2467ef3b2b78 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:02.849121 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398386dd-5f56-42f3-86f4-7739b713c67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:02.849971 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398386dd-5f56-42f3-86f4-7739b713c67e admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:02.851201 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398386dd-5f56-42f3-86f4-7739b713c67e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:02.852093 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398386dd-5f56-42f3-86f4-7739b713c67e admin admin] PATH_INFO: `/v3/projects/5aa68d0b7bc440d2970afce370703bdb/users/06de9b3036c3443fb5af7d949b829fed/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:02.870899 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398386dd-5f56-42f3-86f4-7739b713c67e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5aa68d0b7bc440d2970afce370703bdb, user_id=06de9b3036c3443fb5af7d949b829fed, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:02.873787 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398386dd-5f56-42f3-86f4-7739b713c67e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:02.961375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1055/2110] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:54:02 2022] PUT /identity/v3/projects/5aa68d0b7bc440d2970afce370703bdb/users/06de9b3036c3443fb5af7d949b829fed/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:54:02.965414 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65e5b6-e5d5-432d-9730-806336004151 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:02.976948 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65e5b6-e5d5-432d-9730-806336004151 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:03.036749 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4854e7-af4a-4ae2-a8ab-5a1def7987bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:03.037435 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4854e7-af4a-4ae2-a8ab-5a1def7987bb admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:03.037718 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4854e7-af4a-4ae2-a8ab-5a1def7987bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:03.038004 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4854e7-af4a-4ae2-a8ab-5a1def7987bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:03.038694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4854e7-af4a-4ae2-a8ab-5a1def7987bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:03.040370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4854e7-af4a-4ae2-a8ab-5a1def7987bb admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:03.057270 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1056/2111] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:54:03.061714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398386dd-5f56-42f3-86f4-7739b713c67e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:03.072882 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398386dd-5f56-42f3-86f4-7739b713c67e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:03.077863 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998761e-b2f8-4d41-83ec-a4a307f9b79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:03.078534 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998761e-b2f8-4d41-83ec-a4a307f9b79e admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:03.078821 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998761e-b2f8-4d41-83ec-a4a307f9b79e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:03.079102 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998761e-b2f8-4d41-83ec-a4a307f9b79e admin admin] PATH_INFO: `/v3/projects/5aa68d0b7bc440d2970afce370703bdb/users/06de9b3036c3443fb5af7d949b829fed/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:03.081151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998761e-b2f8-4d41-83ec-a4a307f9b79e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5aa68d0b7bc440d2970afce370703bdb, user_id=06de9b3036c3443fb5af7d949b829fed, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:03.083836 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998761e-b2f8-4d41-83ec-a4a307f9b79e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:03.095977 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1056/2112] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:54:03 2022] PUT /identity/v3/projects/5aa68d0b7bc440d2970afce370703bdb/users/06de9b3036c3443fb5af7d949b829fed/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:03.109325 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f8029c-030d-4e1a-bde0-f3d96c44b4c8 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:03.110184 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f8029c-030d-4e1a-bde0-f3d96c44b4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:03.111072 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f8029c-030d-4e1a-bde0-f3d96c44b4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:03.134939 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:03.143531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-37f8029c-030d-4e1a-bde0-f3d96c44b4c8 None None] MFA Rules not processed for user `06de9b3036c3443fb5af7d949b829fed`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:03.144750 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-37f8029c-030d-4e1a-bde0-f3d96c44b4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:03.219234 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1057/2113] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:03 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:54:03.231149 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998761e-b2f8-4d41-83ec-a4a307f9b79e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:03.233091 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2998761e-b2f8-4d41-83ec-a4a307f9b79e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:03.305650 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998761e-b2f8-4d41-83ec-a4a307f9b79e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:03.309456 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83468d25-6fbc-438b-9139-e66e8bc5534e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:03.311079 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83468d25-6fbc-438b-9139-e66e8bc5534e service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:03.311079 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83468d25-6fbc-438b-9139-e66e8bc5534e service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:03.311079 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83468d25-6fbc-438b-9139-e66e8bc5534e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:03.371146 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83468d25-6fbc-438b-9139-e66e8bc5534e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:03.372767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83468d25-6fbc-438b-9139-e66e8bc5534e service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:03.385769 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1057/2114] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:54:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:54:07.560327 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f8029c-030d-4e1a-bde0-f3d96c44b4c8 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:07.579328 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f8029c-030d-4e1a-bde0-f3d96c44b4c8 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:07.588045 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8441bf1d-7e3f-46c5-b9e3-31c3612b39ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:07.592535 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441bf1d-7e3f-46c5-b9e3-31c3612b39ec service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:07.592535 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441bf1d-7e3f-46c5-b9e3-31c3612b39ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:07.592535 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441bf1d-7e3f-46c5-b9e3-31c3612b39ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:07.687915 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8441bf1d-7e3f-46c5-b9e3-31c3612b39ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:07.690635 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8441bf1d-7e3f-46c5-b9e3-31c3612b39ec service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:07.708060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1058/2115] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:54:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:54:13.379157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=64,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:13.883095 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83468d25-6fbc-438b-9139-e66e8bc5534e service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:13.892914 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83468d25-6fbc-438b-9139-e66e8bc5534e service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:13.897001 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c33e5f8-98e6-491b-8fa7-1eb579ac6d29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:13.901237 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c33e5f8-98e6-491b-8fa7-1eb579ac6d29 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:13.901237 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c33e5f8-98e6-491b-8fa7-1eb579ac6d29 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:13.901237 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c33e5f8-98e6-491b-8fa7-1eb579ac6d29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:13.965434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c33e5f8-98e6-491b-8fa7-1eb579ac6d29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:13.966941 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c33e5f8-98e6-491b-8fa7-1eb579ac6d29 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:13.978001 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1058/2116] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:54:13 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:54:17.699997 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=57 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:23.972937 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=30 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:24.527909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8441bf1d-7e3f-46c5-b9e3-31c3612b39ec service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:24.623371 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8441bf1d-7e3f-46c5-b9e3-31c3612b39ec service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:24.628398 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ecb62d-e845-4c21-a35e-0730ab1ca5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:24.629054 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecb62d-e845-4c21-a35e-0730ab1ca5c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:24.629566 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecb62d-e845-4c21-a35e-0730ab1ca5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:24.629688 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecb62d-e845-4c21-a35e-0730ab1ca5c2 admin admin] PATH_INFO: `/v3/users/ad85ed13d8324a1d8ab49e14f92d0f6a` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:24.630845 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ecb62d-e845-4c21-a35e-0730ab1ca5c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad85ed13d8324a1d8ab49e14f92d0f6a)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:24.632838 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ecb62d-e845-4c21-a35e-0730ab1ca5c2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:24.880584 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-37ecb62d-e845-4c21-a35e-0730ab1ca5c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad85ed13d8324a1d8ab49e14f92d0f6a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:24.881263 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-37ecb62d-e845-4c21-a35e-0730ab1ca5c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad85ed13d8324a1d8ab49e14f92d0f6a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:24.897292 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-37ecb62d-e845-4c21-a35e-0730ab1ca5c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad85ed13d8324a1d8ab49e14f92d0f6a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:24.922983 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-37ecb62d-e845-4c21-a35e-0730ab1ca5c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad85ed13d8324a1d8ab49e14f92d0f6a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:24.930650 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-37ecb62d-e845-4c21-a35e-0730ab1ca5c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad85ed13d8324a1d8ab49e14f92d0f6a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:25.036275 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1059/2117] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:54:24 2022] DELETE /identity/v3/users/ad85ed13d8324a1d8ab49e14f92d0f6a => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:25.041863 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797e053-a0dc-4c73-bc9b-7888dac1367a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:25.042409 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797e053-a0dc-4c73-bc9b-7888dac1367a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:25.042513 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797e053-a0dc-4c73-bc9b-7888dac1367a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:25.064182 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e797e053-a0dc-4c73-bc9b-7888dac1367a None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:25.065480 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e797e053-a0dc-4c73-bc9b-7888dac1367a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:25.143150 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1059/2118] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:54:25.153586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ecb62d-e845-4c21-a35e-0730ab1ca5c2 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:25.155109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-37ecb62d-e845-4c21-a35e-0730ab1ca5c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:25.241776 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ecb62d-e845-4c21-a35e-0730ab1ca5c2 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:25.245749 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a78fe-8453-4fdc-bdf5-30176ae2e7a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:25.246381 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a78fe-8453-4fdc-bdf5-30176ae2e7a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:25.246698 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a78fe-8453-4fdc-bdf5-30176ae2e7a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:25.246968 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a78fe-8453-4fdc-bdf5-30176ae2e7a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:25.258485 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a78fe-8453-4fdc-bdf5-30176ae2e7a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:25.260227 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a78fe-8453-4fdc-bdf5-30176ae2e7a8 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:25.272072 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1060/2119] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:54:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:54:25.848569 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797e053-a0dc-4c73-bc9b-7888dac1367a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:25.850073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e797e053-a0dc-4c73-bc9b-7888dac1367a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:25.864742 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797e053-a0dc-4c73-bc9b-7888dac1367a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:25.870590 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecab6cd-c9ae-446a-912e-58f76698f799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:25.871764 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecab6cd-c9ae-446a-912e-58f76698f799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:25.872241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecab6cd-c9ae-446a-912e-58f76698f799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:25.872688 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecab6cd-c9ae-446a-912e-58f76698f799 admin admin] PATH_INFO: `/v3/projects/79d3d651ad9f465fa235688c586d01cc` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:25.874635 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecab6cd-c9ae-446a-912e-58f76698f799 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79d3d651ad9f465fa235688c586d01cc)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:25.877052 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecab6cd-c9ae-446a-912e-58f76698f799 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:26.064973 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cecab6cd-c9ae-446a-912e-58f76698f799 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79d3d651ad9f465fa235688c586d01cc', 'request_id': 'req-cecab6cd-c9ae-446a-912e-58f76698f799'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:26.074981 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cecab6cd-c9ae-446a-912e-58f76698f799 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79d3d651ad9f465fa235688c586d01cc', 'request_id': 'req-cecab6cd-c9ae-446a-912e-58f76698f799'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:26.202316 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cecab6cd-c9ae-446a-912e-58f76698f799 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79d3d651ad9f465fa235688c586d01cc', 'request_id': 'req-cecab6cd-c9ae-446a-912e-58f76698f799'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:26.226414 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cecab6cd-c9ae-446a-912e-58f76698f799 admin admin] The token cache is being invalidate because project 79d3d651ad9f465fa235688c586d01cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:54:26.226910 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cecab6cd-c9ae-446a-912e-58f76698f799 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:26.227755 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cecab6cd-c9ae-446a-912e-58f76698f799 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:26.230900 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1060/2120] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:54:25 2022] DELETE /identity/v3/projects/79d3d651ad9f465fa235688c586d01cc => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:26.235770 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a78fe-8453-4fdc-bdf5-30176ae2e7a8 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:26.256253 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a78fe-8453-4fdc-bdf5-30176ae2e7a8 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:26.289775 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:26.375542 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e89a2-da71-485a-a410-8d7d482e072e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:26.376973 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e89a2-da71-485a-a410-8d7d482e072e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:26.377262 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e89a2-da71-485a-a410-8d7d482e072e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:26.377541 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e89a2-da71-485a-a410-8d7d482e072e admin admin] PATH_INFO: `/v3/users/06de9b3036c3443fb5af7d949b829fed` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:26.379629 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e89a2-da71-485a-a410-8d7d482e072e admin admin] RBAC: Authorizing `identity:delete_user(user_id=06de9b3036c3443fb5af7d949b829fed)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:26.388827 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e89a2-da71-485a-a410-8d7d482e072e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:26.739880 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3b1e89a2-da71-485a-a410-8d7d482e072e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06de9b3036c3443fb5af7d949b829fed'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:26.740681 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3b1e89a2-da71-485a-a410-8d7d482e072e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06de9b3036c3443fb5af7d949b829fed'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:26.755974 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3b1e89a2-da71-485a-a410-8d7d482e072e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06de9b3036c3443fb5af7d949b829fed'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:26.764113 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3b1e89a2-da71-485a-a410-8d7d482e072e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06de9b3036c3443fb5af7d949b829fed'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:26.767863 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3b1e89a2-da71-485a-a410-8d7d482e072e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06de9b3036c3443fb5af7d949b829fed'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:26.865616 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1061/2121] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:54:26 2022] DELETE /identity/v3/users/06de9b3036c3443fb5af7d949b829fed => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:27.388424 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecab6cd-c9ae-446a-912e-58f76698f799 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:27.390628 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-cecab6cd-c9ae-446a-912e-58f76698f799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:27.404606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecab6cd-c9ae-446a-912e-58f76698f799 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:27.472523 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275963f-6294-4e05-8490-33e99702b193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:27.473248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275963f-6294-4e05-8490-33e99702b193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:27.473695 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275963f-6294-4e05-8490-33e99702b193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:27.474020 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275963f-6294-4e05-8490-33e99702b193 admin admin] PATH_INFO: `/v3/projects/5aa68d0b7bc440d2970afce370703bdb` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:27.475465 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275963f-6294-4e05-8490-33e99702b193 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5aa68d0b7bc440d2970afce370703bdb)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:27.477534 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275963f-6294-4e05-8490-33e99702b193 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:27.615894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:27.639242 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f275963f-6294-4e05-8490-33e99702b193 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5aa68d0b7bc440d2970afce370703bdb', 'request_id': 'req-f275963f-6294-4e05-8490-33e99702b193'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:27.651457 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f275963f-6294-4e05-8490-33e99702b193 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5aa68d0b7bc440d2970afce370703bdb', 'request_id': 'req-f275963f-6294-4e05-8490-33e99702b193'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:27.727834 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f275963f-6294-4e05-8490-33e99702b193 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5aa68d0b7bc440d2970afce370703bdb', 'request_id': 'req-f275963f-6294-4e05-8490-33e99702b193'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:27.741173 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f275963f-6294-4e05-8490-33e99702b193 admin admin] The token cache is being invalidate because project 5aa68d0b7bc440d2970afce370703bdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:54:27.741466 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f275963f-6294-4e05-8490-33e99702b193 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:27.742136 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f275963f-6294-4e05-8490-33e99702b193 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:27.743809 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1061/2122] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:54:27 2022] DELETE /identity/v3/projects/5aa68d0b7bc440d2970afce370703bdb => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:54:27.749681 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1c751-01b2-496e-9031-4672b02d3dda None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:27.750153 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1c751-01b2-496e-9031-4672b02d3dda None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:27.750584 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1c751-01b2-496e-9031-4672b02d3dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:27.775687 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-40f1c751-01b2-496e-9031-4672b02d3dda None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:27.776912 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-40f1c751-01b2-496e-9031-4672b02d3dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:27.857909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1062/2123] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:54:27.866253 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e88d5c-d07d-445d-a45e-2194af392825 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:27.866581 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e88d5c-d07d-445d-a45e-2194af392825 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:27.866874 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e88d5c-d07d-445d-a45e-2194af392825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:27.889501 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-69e88d5c-d07d-445d-a45e-2194af392825 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:27.890778 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-69e88d5c-d07d-445d-a45e-2194af392825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:27.898990 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1062/2124] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:54:27.903333 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1c751-01b2-496e-9031-4672b02d3dda None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:27.914140 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1c751-01b2-496e-9031-4672b02d3dda None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:27.919054 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f321f1a-f768-4ff8-b6a0-cc3f1f78072b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:27.919704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f321f1a-f768-4ff8-b6a0-cc3f1f78072b admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:27.919990 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f321f1a-f768-4ff8-b6a0-cc3f1f78072b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:27.920271 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f321f1a-f768-4ff8-b6a0-cc3f1f78072b admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:27.926546 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f321f1a-f768-4ff8-b6a0-cc3f1f78072b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:54:27.926546 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f321f1a-f768-4ff8-b6a0-cc3f1f78072b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:27.926546 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f321f1a-f768-4ff8-b6a0-cc3f1f78072b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:27.935684 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1063/2125] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:54:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:54:27.940618 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e88d5c-d07d-445d-a45e-2194af392825 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:27.950185 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e88d5c-d07d-445d-a45e-2194af392825 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:27.956465 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb9ac0-f9d1-4fba-9562-ec93714985de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:27.957091 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb9ac0-f9d1-4fba-9562-ec93714985de admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:27.957377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb9ac0-f9d1-4fba-9562-ec93714985de admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:27.957654 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb9ac0-f9d1-4fba-9562-ec93714985de admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:27.958337 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb9ac0-f9d1-4fba-9562-ec93714985de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:27.960299 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb9ac0-f9d1-4fba-9562-ec93714985de admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:28.090070 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1063/2126] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:54:27 2022] POST /identity/v3/projects => generated 373 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:54:28.097181 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f321f1a-f768-4ff8-b6a0-cc3f1f78072b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:28.107581 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f321f1a-f768-4ff8-b6a0-cc3f1f78072b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:28.134945 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:28.167159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a138eb8-3c0e-4e8c-ba0f-949236cebdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:28.167819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a138eb8-3c0e-4e8c-ba0f-949236cebdd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:28.168119 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a138eb8-3c0e-4e8c-ba0f-949236cebdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:28.168388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a138eb8-3c0e-4e8c-ba0f-949236cebdd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:28.169067 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a138eb8-3c0e-4e8c-ba0f-949236cebdd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:28.171550 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a138eb8-3c0e-4e8c-ba0f-949236cebdd3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:28.253167 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1064/2127] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:54:28 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:54:28.255858 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb9ac0-f9d1-4fba-9562-ec93714985de admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:28.273291 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb9ac0-f9d1-4fba-9562-ec93714985de admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:28.282100 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d6c8d5-6925-4ce1-87f3-0366e8cdc7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:28.283626 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6c8d5-6925-4ce1-87f3-0366e8cdc7a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:28.283727 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6c8d5-6925-4ce1-87f3-0366e8cdc7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:28.284147 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6c8d5-6925-4ce1-87f3-0366e8cdc7a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:28.285380 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d6c8d5-6925-4ce1-87f3-0366e8cdc7a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:28.288439 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d6c8d5-6925-4ce1-87f3-0366e8cdc7a4 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:28.307389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1064/2128] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:28 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:54:28.312396 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a138eb8-3c0e-4e8c-ba0f-949236cebdd3 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:28.324731 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a138eb8-3c0e-4e8c-ba0f-949236cebdd3 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:28.329640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256502e-6384-4d4a-80af-d44e0e67854b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:28.330426 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256502e-6384-4d4a-80af-d44e0e67854b admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:28.330711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256502e-6384-4d4a-80af-d44e0e67854b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:28.331026 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256502e-6384-4d4a-80af-d44e0e67854b admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:28.331867 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256502e-6384-4d4a-80af-d44e0e67854b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:28.339671 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256502e-6384-4d4a-80af-d44e0e67854b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:28.344358 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1065/2129] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:54:28.348825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d6c8d5-6925-4ce1-87f3-0366e8cdc7a4 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:28.360145 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d6c8d5-6925-4ce1-87f3-0366e8cdc7a4 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:28.365491 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c2611-820d-4096-9781-f95f0911422a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:28.366170 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c2611-820d-4096-9781-f95f0911422a admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:28.366454 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c2611-820d-4096-9781-f95f0911422a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:28.366736 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c2611-820d-4096-9781-f95f0911422a admin admin] PATH_INFO: `/v3/projects/bd4e47dfa8ff43829da20aa31c4bb25b/users/81d4b47691f44cab80c244a0d21542db/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:28.384760 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c2611-820d-4096-9781-f95f0911422a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd4e47dfa8ff43829da20aa31c4bb25b, user_id=81d4b47691f44cab80c244a0d21542db, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:28.387727 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c2611-820d-4096-9781-f95f0911422a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:28.565133 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1065/2130] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:54:28 2022] PUT /identity/v3/projects/bd4e47dfa8ff43829da20aa31c4bb25b/users/81d4b47691f44cab80c244a0d21542db/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:28.651645 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716854e-2f3e-485a-9144-57a789d7b09f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:28.657999 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716854e-2f3e-485a-9144-57a789d7b09f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:28.657999 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716854e-2f3e-485a-9144-57a789d7b09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:28.680536 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c716854e-2f3e-485a-9144-57a789d7b09f None None] MFA Rules not processed for user `81d4b47691f44cab80c244a0d21542db`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:28.681760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c716854e-2f3e-485a-9144-57a789d7b09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:28.752618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1066/2131] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:54:28.772732 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c2611-820d-4096-9781-f95f0911422a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:28.773059 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b93c2611-820d-4096-9781-f95f0911422a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:28.838476 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c2611-820d-4096-9781-f95f0911422a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:28.842577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d03a2-5b66-4159-9396-bfa47680d219 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:28.843309 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d03a2-5b66-4159-9396-bfa47680d219 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:28.843516 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d03a2-5b66-4159-9396-bfa47680d219 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:28.844348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d03a2-5b66-4159-9396-bfa47680d219 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:28.854585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d03a2-5b66-4159-9396-bfa47680d219 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:28.856102 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d03a2-5b66-4159-9396-bfa47680d219 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:28.868394 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1066/2132] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:54:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:54:29.915203 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716854e-2f3e-485a-9144-57a789d7b09f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:29.917494 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c716854e-2f3e-485a-9144-57a789d7b09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:29.991537 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716854e-2f3e-485a-9144-57a789d7b09f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:29.996158 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673cb41f-3186-40df-85e0-8af6a857a180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:29.996792 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673cb41f-3186-40df-85e0-8af6a857a180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:29.997069 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673cb41f-3186-40df-85e0-8af6a857a180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:29.997341 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673cb41f-3186-40df-85e0-8af6a857a180 admin admin] PATH_INFO: `/v3/users/aacb04366c3a4eca9aadd56ea31b7acb` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:29.998495 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673cb41f-3186-40df-85e0-8af6a857a180 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aacb04366c3a4eca9aadd56ea31b7acb)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:30.001057 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673cb41f-3186-40df-85e0-8af6a857a180 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:30.224840 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d03a2-5b66-4159-9396-bfa47680d219 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:30.228072 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-877d03a2-5b66-4159-9396-bfa47680d219 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:30.231654 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:30.241334 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-673cb41f-3186-40df-85e0-8af6a857a180 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aacb04366c3a4eca9aadd56ea31b7acb'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:30.242555 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-673cb41f-3186-40df-85e0-8af6a857a180 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aacb04366c3a4eca9aadd56ea31b7acb'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:30.250820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d03a2-5b66-4159-9396-bfa47680d219 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:30.257285 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb991f38-13f6-43af-9c29-4ecff7b64428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:30.258406 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb991f38-13f6-43af-9c29-4ecff7b64428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:30.259068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb991f38-13f6-43af-9c29-4ecff7b64428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:30.259615 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb991f38-13f6-43af-9c29-4ecff7b64428 admin admin] PATH_INFO: `/v3/users/1e12980e507e4b148561ecf82580f377` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:30.260199 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-673cb41f-3186-40df-85e0-8af6a857a180 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aacb04366c3a4eca9aadd56ea31b7acb'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:30.261062 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb991f38-13f6-43af-9c29-4ecff7b64428 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e12980e507e4b148561ecf82580f377)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:30.264071 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb991f38-13f6-43af-9c29-4ecff7b64428 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:30.268668 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-673cb41f-3186-40df-85e0-8af6a857a180 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aacb04366c3a4eca9aadd56ea31b7acb'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:30.271295 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-673cb41f-3186-40df-85e0-8af6a857a180 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aacb04366c3a4eca9aadd56ea31b7acb'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:30.362876 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1067/2133] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:54:29 2022] DELETE /identity/v3/users/aacb04366c3a4eca9aadd56ea31b7acb => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:30.367218 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d28ad-4095-47eb-99b0-c8ad06cbc2a9 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:30.367546 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d28ad-4095-47eb-99b0-c8ad06cbc2a9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:30.367830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d28ad-4095-47eb-99b0-c8ad06cbc2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:30.388721 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-7c6d28ad-4095-47eb-99b0-c8ad06cbc2a9 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:30.389946 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7c6d28ad-4095-47eb-99b0-c8ad06cbc2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:30.450606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1068/2134] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:54:30.460545 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d28ad-4095-47eb-99b0-c8ad06cbc2a9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:30.461996 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7c6d28ad-4095-47eb-99b0-c8ad06cbc2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:30.505088 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-eb991f38-13f6-43af-9c29-4ecff7b64428 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e12980e507e4b148561ecf82580f377'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:30.505815 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-eb991f38-13f6-43af-9c29-4ecff7b64428 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e12980e507e4b148561ecf82580f377'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:30.507206 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:30.522737 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-eb991f38-13f6-43af-9c29-4ecff7b64428 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e12980e507e4b148561ecf82580f377'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:30.524931 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d28ad-4095-47eb-99b0-c8ad06cbc2a9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:30.529324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e0678-5e8b-47a6-9158-e303e5dd89ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:30.530578 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-eb991f38-13f6-43af-9c29-4ecff7b64428 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e12980e507e4b148561ecf82580f377'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:30.530766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e0678-5e8b-47a6-9158-e303e5dd89ea service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:30.530766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e0678-5e8b-47a6-9158-e303e5dd89ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:30.531076 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-eb991f38-13f6-43af-9c29-4ecff7b64428 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e12980e507e4b148561ecf82580f377'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:30.531234 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e0678-5e8b-47a6-9158-e303e5dd89ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:30.533321 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-886e0678-5e8b-47a6-9158-e303e5dd89ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:30.547019 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e0678-5e8b-47a6-9158-e303e5dd89ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:30.548542 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e0678-5e8b-47a6-9158-e303e5dd89ea service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:30.558999 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1069/2135] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:54:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:54:30.622603 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1067/2136] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:54:30 2022] DELETE /identity/v3/users/1e12980e507e4b148561ecf82580f377 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:54:30.630369 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a7457-b4bd-49a7-9dd4-6d0ef489211f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:30.630864 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a7457-b4bd-49a7-9dd4-6d0ef489211f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:30.631324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a7457-b4bd-49a7-9dd4-6d0ef489211f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:30.661148 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0b2a7457-b4bd-49a7-9dd4-6d0ef489211f None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:30.662923 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0b2a7457-b4bd-49a7-9dd4-6d0ef489211f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:30.748289 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:30.769925 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1070/2137] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:54:30.791682 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb991f38-13f6-43af-9c29-4ecff7b64428 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:30.793770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-eb991f38-13f6-43af-9c29-4ecff7b64428 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:30.925849 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb991f38-13f6-43af-9c29-4ecff7b64428 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:30.929913 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1bcdde-bf1d-4e57-880c-7b38a389ac6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:30.930542 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bcdde-bf1d-4e57-880c-7b38a389ac6c service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:30.930849 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bcdde-bf1d-4e57-880c-7b38a389ac6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:30.931150 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bcdde-bf1d-4e57-880c-7b38a389ac6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:30.942392 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1bcdde-bf1d-4e57-880c-7b38a389ac6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:30.944051 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1bcdde-bf1d-4e57-880c-7b38a389ac6c service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:30.955038 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1068/2138] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:54:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:54:31.155908 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2a7457-b4bd-49a7-9dd4-6d0ef489211f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:31.157782 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0b2a7457-b4bd-49a7-9dd4-6d0ef489211f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:31.170324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2a7457-b4bd-49a7-9dd4-6d0ef489211f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:31.176518 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1781c3d3-ddcd-4c90-98a5-20a2fb68b8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:31.177177 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781c3d3-ddcd-4c90-98a5-20a2fb68b8de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:31.177665 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781c3d3-ddcd-4c90-98a5-20a2fb68b8de admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:31.177938 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781c3d3-ddcd-4c90-98a5-20a2fb68b8de admin admin] PATH_INFO: `/v3/projects/5e97a678d3d1434387855e553c35e55a` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:31.179670 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1781c3d3-ddcd-4c90-98a5-20a2fb68b8de admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e97a678d3d1434387855e553c35e55a)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:31.181601 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1781c3d3-ddcd-4c90-98a5-20a2fb68b8de admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:31.234393 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1bcdde-bf1d-4e57-880c-7b38a389ac6c service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:31.234393 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ca1bcdde-bf1d-4e57-880c-7b38a389ac6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:31.245749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1bcdde-bf1d-4e57-880c-7b38a389ac6c service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:31.251028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cf63d-8d96-4da9-a88a-3d2130a074d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:31.251719 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cf63d-8d96-4da9-a88a-3d2130a074d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:31.252001 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cf63d-8d96-4da9-a88a-3d2130a074d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:31.252306 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cf63d-8d96-4da9-a88a-3d2130a074d0 admin admin] PATH_INFO: `/v3/users/645318a5f0164a32ab559213aa67462d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:31.253462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386cf63d-8d96-4da9-a88a-3d2130a074d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=645318a5f0164a32ab559213aa67462d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:31.255451 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386cf63d-8d96-4da9-a88a-3d2130a074d0 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:31.337274 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1781c3d3-ddcd-4c90-98a5-20a2fb68b8de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e97a678d3d1434387855e553c35e55a', 'request_id': 'req-1781c3d3-ddcd-4c90-98a5-20a2fb68b8de'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:31.343849 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1781c3d3-ddcd-4c90-98a5-20a2fb68b8de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e97a678d3d1434387855e553c35e55a', 'request_id': 'req-1781c3d3-ddcd-4c90-98a5-20a2fb68b8de'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:31.815266 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1781c3d3-ddcd-4c90-98a5-20a2fb68b8de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e97a678d3d1434387855e553c35e55a', 'request_id': 'req-1781c3d3-ddcd-4c90-98a5-20a2fb68b8de'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:31.842399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1781c3d3-ddcd-4c90-98a5-20a2fb68b8de admin admin] The token cache is being invalidate because project 5e97a678d3d1434387855e553c35e55a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:54:31.842927 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1781c3d3-ddcd-4c90-98a5-20a2fb68b8de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:31.844412 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1781c3d3-ddcd-4c90-98a5-20a2fb68b8de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:31.847284 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1071/2139] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:54:31 2022] DELETE /identity/v3/projects/5e97a678d3d1434387855e553c35e55a => generated 0 bytes in 692 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:31.852721 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1781c3d3-ddcd-4c90-98a5-20a2fb68b8de admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:31.855565 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1781c3d3-ddcd-4c90-98a5-20a2fb68b8de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:31.955093 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1781c3d3-ddcd-4c90-98a5-20a2fb68b8de admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:31.966975 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc641f49-f636-426e-91f3-292558ef1e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:31.967698 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc641f49-f636-426e-91f3-292558ef1e6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:31.968045 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc641f49-f636-426e-91f3-292558ef1e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:31.969370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc641f49-f636-426e-91f3-292558ef1e6c admin admin] PATH_INFO: `/v3/users/db3824759bb3467583616967d9e66db7` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:31.969942 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc641f49-f636-426e-91f3-292558ef1e6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=db3824759bb3467583616967d9e66db7)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:31.972092 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc641f49-f636-426e-91f3-292558ef1e6c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:32.080470 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-386cf63d-8d96-4da9-a88a-3d2130a074d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '645318a5f0164a32ab559213aa67462d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:32.081879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-386cf63d-8d96-4da9-a88a-3d2130a074d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '645318a5f0164a32ab559213aa67462d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:32.104910 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-386cf63d-8d96-4da9-a88a-3d2130a074d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '645318a5f0164a32ab559213aa67462d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:32.118500 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-386cf63d-8d96-4da9-a88a-3d2130a074d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '645318a5f0164a32ab559213aa67462d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:32.119313 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-386cf63d-8d96-4da9-a88a-3d2130a074d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '645318a5f0164a32ab559213aa67462d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:32.241018 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1069/2140] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:54:31 2022] DELETE /identity/v3/users/645318a5f0164a32ab559213aa67462d => generated 0 bytes in 1011 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:32.242412 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cf63d-8d96-4da9-a88a-3d2130a074d0 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:32.244506 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-386cf63d-8d96-4da9-a88a-3d2130a074d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:32.299763 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:32.320941 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cf63d-8d96-4da9-a88a-3d2130a074d0 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:32.325911 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117561c-f36b-4c62-94ca-c2fca431021a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:32.326549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117561c-f36b-4c62-94ca-c2fca431021a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:32.326835 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117561c-f36b-4c62-94ca-c2fca431021a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:32.327117 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117561c-f36b-4c62-94ca-c2fca431021a admin admin] PATH_INFO: `/v3/projects/abae4f3354b84328a96144d093dd17be` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:32.328547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117561c-f36b-4c62-94ca-c2fca431021a admin admin] RBAC: Authorizing `identity:delete_project(project_id=abae4f3354b84328a96144d093dd17be)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:32.330586 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117561c-f36b-4c62-94ca-c2fca431021a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:32.341765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cc641f49-f636-426e-91f3-292558ef1e6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db3824759bb3467583616967d9e66db7'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:32.342498 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cc641f49-f636-426e-91f3-292558ef1e6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db3824759bb3467583616967d9e66db7'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:32.356428 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cc641f49-f636-426e-91f3-292558ef1e6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db3824759bb3467583616967d9e66db7'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:32.364057 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cc641f49-f636-426e-91f3-292558ef1e6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db3824759bb3467583616967d9e66db7'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:32.364817 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cc641f49-f636-426e-91f3-292558ef1e6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db3824759bb3467583616967d9e66db7'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:32.446993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1072/2141] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:54:31 2022] DELETE /identity/v3/users/db3824759bb3467583616967d9e66db7 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:32.446993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9ac25-3cbd-4d89-88d7-522d454eaba4 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:32.446993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9ac25-3cbd-4d89-88d7-522d454eaba4 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:32.446993 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9ac25-3cbd-4d89-88d7-522d454eaba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:32.455344 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:32.463111 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2117561c-f36b-4c62-94ca-c2fca431021a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abae4f3354b84328a96144d093dd17be', 'request_id': 'req-2117561c-f36b-4c62-94ca-c2fca431021a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:32.470272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2117561c-f36b-4c62-94ca-c2fca431021a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abae4f3354b84328a96144d093dd17be', 'request_id': 'req-2117561c-f36b-4c62-94ca-c2fca431021a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:32.476301 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6bd9ac25-3cbd-4d89-88d7-522d454eaba4 None None] MFA Rules not processed for user `81d4b47691f44cab80c244a0d21542db`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:32.476301 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6bd9ac25-3cbd-4d89-88d7-522d454eaba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:32.545645 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1073/2142] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:54:32.547659 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbea45-5f02-484a-a57e-c18ce604ecb3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:32.547947 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbea45-5f02-484a-a57e-c18ce604ecb3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:32.548221 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbea45-5f02-484a-a57e-c18ce604ecb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:32.570446 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-39dbea45-5f02-484a-a57e-c18ce604ecb3 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:32.571770 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-39dbea45-5f02-484a-a57e-c18ce604ecb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:32.579967 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2117561c-f36b-4c62-94ca-c2fca431021a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abae4f3354b84328a96144d093dd17be', 'request_id': 'req-2117561c-f36b-4c62-94ca-c2fca431021a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:32.599127 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2117561c-f36b-4c62-94ca-c2fca431021a admin admin] The token cache is being invalidate because project abae4f3354b84328a96144d093dd17be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:54:32.599804 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2117561c-f36b-4c62-94ca-c2fca431021a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:32.600668 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2117561c-f36b-4c62-94ca-c2fca431021a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:32.603038 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1070/2143] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:54:32 2022] DELETE /identity/v3/projects/abae4f3354b84328a96144d093dd17be => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:32.607221 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117561c-f36b-4c62-94ca-c2fca431021a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:32.610517 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2117561c-f36b-4c62-94ca-c2fca431021a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:32.649546 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1074/2144] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:54:32.665058 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dbea45-5f02-484a-a57e-c18ce604ecb3 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:32.667489 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-39dbea45-5f02-484a-a57e-c18ce604ecb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:32.717770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117561c-f36b-4c62-94ca-c2fca431021a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:32.722821 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb58f2-0264-4478-8710-5da0abcef7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:32.724071 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb58f2-0264-4478-8710-5da0abcef7f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:32.724364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb58f2-0264-4478-8710-5da0abcef7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:32.724650 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb58f2-0264-4478-8710-5da0abcef7f1 admin admin] PATH_INFO: `/v3/users/71cbf8bc9cee4dc6bde3b4d886224c12` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:32.725934 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb58f2-0264-4478-8710-5da0abcef7f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71cbf8bc9cee4dc6bde3b4d886224c12)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:32.729672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb58f2-0264-4478-8710-5da0abcef7f1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:32.754421 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dbea45-5f02-484a-a57e-c18ce604ecb3 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:32.760141 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a8773-bfdf-45ad-a3cc-442fcc702c88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:32.761660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a8773-bfdf-45ad-a3cc-442fcc702c88 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:32.762183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a8773-bfdf-45ad-a3cc-442fcc702c88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:32.762678 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a8773-bfdf-45ad-a3cc-442fcc702c88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:32.775907 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64a8773-bfdf-45ad-a3cc-442fcc702c88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:32.778328 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64a8773-bfdf-45ad-a3cc-442fcc702c88 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:32.789826 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1075/2145] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:54:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:54:33.005661 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-75bb58f2-0264-4478-8710-5da0abcef7f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71cbf8bc9cee4dc6bde3b4d886224c12'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:33.007016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-75bb58f2-0264-4478-8710-5da0abcef7f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71cbf8bc9cee4dc6bde3b4d886224c12'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:33.023294 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-75bb58f2-0264-4478-8710-5da0abcef7f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71cbf8bc9cee4dc6bde3b4d886224c12'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:33.033032 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:33.037735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-75bb58f2-0264-4478-8710-5da0abcef7f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71cbf8bc9cee4dc6bde3b4d886224c12'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:33.038974 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-75bb58f2-0264-4478-8710-5da0abcef7f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71cbf8bc9cee4dc6bde3b4d886224c12'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:33.095608 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a8773-bfdf-45ad-a3cc-442fcc702c88 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:33.101155 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a64a8773-bfdf-45ad-a3cc-442fcc702c88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:33.112844 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a8773-bfdf-45ad-a3cc-442fcc702c88 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:33.118192 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a3326-a7ff-4509-89a5-ea21d4500aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:33.119441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a3326-a7ff-4509-89a5-ea21d4500aff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:33.119441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a3326-a7ff-4509-89a5-ea21d4500aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:33.119820 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a3326-a7ff-4509-89a5-ea21d4500aff admin admin] PATH_INFO: `/v3/projects/446ab7d8cce84609b469dff72678ed91` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:33.121331 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a3326-a7ff-4509-89a5-ea21d4500aff admin admin] RBAC: Authorizing `identity:delete_project(project_id=446ab7d8cce84609b469dff72678ed91)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:33.125310 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a3326-a7ff-4509-89a5-ea21d4500aff admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:33.142471 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1071/2146] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:54:32 2022] DELETE /identity/v3/users/71cbf8bc9cee4dc6bde3b4d886224c12 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:33.305197 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:33.308770 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f55a3326-a7ff-4509-89a5-ea21d4500aff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '446ab7d8cce84609b469dff72678ed91', 'request_id': 'req-f55a3326-a7ff-4509-89a5-ea21d4500aff'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:33.315684 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f55a3326-a7ff-4509-89a5-ea21d4500aff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '446ab7d8cce84609b469dff72678ed91', 'request_id': 'req-f55a3326-a7ff-4509-89a5-ea21d4500aff'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:33.421739 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f55a3326-a7ff-4509-89a5-ea21d4500aff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '446ab7d8cce84609b469dff72678ed91', 'request_id': 'req-f55a3326-a7ff-4509-89a5-ea21d4500aff'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:33.459732 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f55a3326-a7ff-4509-89a5-ea21d4500aff admin admin] The token cache is being invalidate because project 446ab7d8cce84609b469dff72678ed91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:54:33.461845 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f55a3326-a7ff-4509-89a5-ea21d4500aff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:33.464596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f55a3326-a7ff-4509-89a5-ea21d4500aff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:33.468520 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1076/2147] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:54:33 2022] DELETE /identity/v3/projects/446ab7d8cce84609b469dff72678ed91 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:54:33.473487 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb58f2-0264-4478-8710-5da0abcef7f1 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:33.475333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-75bb58f2-0264-4478-8710-5da0abcef7f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:33.590950 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb58f2-0264-4478-8710-5da0abcef7f1 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:33.631559 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a3326-a7ff-4509-89a5-ea21d4500aff admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:33.633828 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f55a3326-a7ff-4509-89a5-ea21d4500aff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:33.672884 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c65c92-1edc-4bec-ac88-0c9b05cc526a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:33.673550 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c65c92-1edc-4bec-ac88-0c9b05cc526a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:33.673878 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c65c92-1edc-4bec-ac88-0c9b05cc526a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:33.674240 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c65c92-1edc-4bec-ac88-0c9b05cc526a admin admin] PATH_INFO: `/v3/users/0c7f376d95ec4d1db0079c448e7b24b7` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:33.675337 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c65c92-1edc-4bec-ac88-0c9b05cc526a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c7f376d95ec4d1db0079c448e7b24b7)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:33.677339 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c65c92-1edc-4bec-ac88-0c9b05cc526a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:33.744032 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a3326-a7ff-4509-89a5-ea21d4500aff admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:33.744540 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849f69d-d651-46dc-b027-61220274a782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:33.745042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849f69d-d651-46dc-b027-61220274a782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:33.745329 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849f69d-d651-46dc-b027-61220274a782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:33.745606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849f69d-d651-46dc-b027-61220274a782 admin admin] PATH_INFO: `/v3/projects/d203b76aa92244d48e32dbc9e43b3cc8` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:33.746782 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e849f69d-d651-46dc-b027-61220274a782 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d203b76aa92244d48e32dbc9e43b3cc8)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:33.748786 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e849f69d-d651-46dc-b027-61220274a782 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:33.908416 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e849f69d-d651-46dc-b027-61220274a782 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd203b76aa92244d48e32dbc9e43b3cc8', 'request_id': 'req-e849f69d-d651-46dc-b027-61220274a782'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:33.915314 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b2c65c92-1edc-4bec-ac88-0c9b05cc526a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c7f376d95ec4d1db0079c448e7b24b7'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:33.916906 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e849f69d-d651-46dc-b027-61220274a782 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd203b76aa92244d48e32dbc9e43b3cc8', 'request_id': 'req-e849f69d-d651-46dc-b027-61220274a782'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:33.917135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b2c65c92-1edc-4bec-ac88-0c9b05cc526a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c7f376d95ec4d1db0079c448e7b24b7'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:33.932366 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b2c65c92-1edc-4bec-ac88-0c9b05cc526a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c7f376d95ec4d1db0079c448e7b24b7'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:33.939814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b2c65c92-1edc-4bec-ac88-0c9b05cc526a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c7f376d95ec4d1db0079c448e7b24b7'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:33.940573 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b2c65c92-1edc-4bec-ac88-0c9b05cc526a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c7f376d95ec4d1db0079c448e7b24b7'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:34.019005 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e849f69d-d651-46dc-b027-61220274a782 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd203b76aa92244d48e32dbc9e43b3cc8', 'request_id': 'req-e849f69d-d651-46dc-b027-61220274a782'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:34.030053 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e849f69d-d651-46dc-b027-61220274a782 admin admin] The token cache is being invalidate because project d203b76aa92244d48e32dbc9e43b3cc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:54:34.030457 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e849f69d-d651-46dc-b027-61220274a782 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:34.031797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e849f69d-d651-46dc-b027-61220274a782 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:34.034415 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1077/2148] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:54:33 2022] DELETE /identity/v3/projects/d203b76aa92244d48e32dbc9e43b3cc8 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:54:34.035336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849f69d-d651-46dc-b027-61220274a782 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:34.036799 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e849f69d-d651-46dc-b027-61220274a782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:34.113211 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849f69d-d651-46dc-b027-61220274a782 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:34.117970 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815aeb4-0dc1-4a2d-84e6-5b701811c02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:34.118614 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815aeb4-0dc1-4a2d-84e6-5b701811c02e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:34.118891 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815aeb4-0dc1-4a2d-84e6-5b701811c02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:34.119163 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815aeb4-0dc1-4a2d-84e6-5b701811c02e admin admin] PATH_INFO: `/v3/projects/b3623655d8da453183a7678ea072564e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:34.120341 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4815aeb4-0dc1-4a2d-84e6-5b701811c02e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3623655d8da453183a7678ea072564e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:34.122232 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4815aeb4-0dc1-4a2d-84e6-5b701811c02e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:34.261269 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1072/2149] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:54:33 2022] DELETE /identity/v3/users/0c7f376d95ec4d1db0079c448e7b24b7 => generated 0 bytes in 788 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:34.262712 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c65c92-1edc-4bec-ac88-0c9b05cc526a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:34.264698 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b2c65c92-1edc-4bec-ac88-0c9b05cc526a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:34.286943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:34.337853 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c65c92-1edc-4bec-ac88-0c9b05cc526a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:34.343920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4815aeb4-0dc1-4a2d-84e6-5b701811c02e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3623655d8da453183a7678ea072564e', 'request_id': 'req-4815aeb4-0dc1-4a2d-84e6-5b701811c02e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:34.353717 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4815aeb4-0dc1-4a2d-84e6-5b701811c02e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3623655d8da453183a7678ea072564e', 'request_id': 'req-4815aeb4-0dc1-4a2d-84e6-5b701811c02e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:34.374094 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:34.398851 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796afff6-3730-49ce-a48b-75a093f0dcfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:34.399697 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796afff6-3730-49ce-a48b-75a093f0dcfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:34.400047 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796afff6-3730-49ce-a48b-75a093f0dcfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:34.400518 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796afff6-3730-49ce-a48b-75a093f0dcfe admin admin] PATH_INFO: `/v3/users/a408f63dca4f4392bb176a50c0d72248` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:34.401868 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796afff6-3730-49ce-a48b-75a093f0dcfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=a408f63dca4f4392bb176a50c0d72248)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:34.404142 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796afff6-3730-49ce-a48b-75a093f0dcfe admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:34.427835 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4815aeb4-0dc1-4a2d-84e6-5b701811c02e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3623655d8da453183a7678ea072564e', 'request_id': 'req-4815aeb4-0dc1-4a2d-84e6-5b701811c02e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:34.450549 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4815aeb4-0dc1-4a2d-84e6-5b701811c02e admin admin] The token cache is being invalidate because project b3623655d8da453183a7678ea072564e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:54:34.451458 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4815aeb4-0dc1-4a2d-84e6-5b701811c02e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:34.453569 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4815aeb4-0dc1-4a2d-84e6-5b701811c02e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:34.456949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1078/2150] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:54:34 2022] DELETE /identity/v3/projects/b3623655d8da453183a7678ea072564e => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:34.466400 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b3e35-642d-4cba-a039-82084ad6e5f2 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:34.467166 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b3e35-642d-4cba-a039-82084ad6e5f2 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:34.472219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b3e35-642d-4cba-a039-82084ad6e5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:34.511429 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0b2b3e35-642d-4cba-a039-82084ad6e5f2 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:34.512779 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0b2b3e35-642d-4cba-a039-82084ad6e5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:34.522113 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1079/2151] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:54:34.530184 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7029a1-3e6f-4595-9da5-da1067412c2f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:34.531291 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7029a1-3e6f-4595-9da5-da1067412c2f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:34.531663 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7029a1-3e6f-4595-9da5-da1067412c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:34.551200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6c7029a1-3e6f-4595-9da5-da1067412c2f None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:34.552468 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6c7029a1-3e6f-4595-9da5-da1067412c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:34.562637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1080/2152] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:54:34.567770 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7029a1-3e6f-4595-9da5-da1067412c2f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:34.580527 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7029a1-3e6f-4595-9da5-da1067412c2f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:34.588280 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46198f-d355-4b32-bc89-5c19170c28c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:34.590161 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46198f-d355-4b32-bc89-5c19170c28c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:34.590673 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46198f-d355-4b32-bc89-5c19170c28c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:34.591067 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46198f-d355-4b32-bc89-5c19170c28c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:34.591980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa46198f-d355-4b32-bc89-5c19170c28c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:54:34.592394 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa46198f-d355-4b32-bc89-5c19170c28c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:34.594933 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa46198f-d355-4b32-bc89-5c19170c28c5 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:34.615924 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1081/2153] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:54:34.621214 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46198f-d355-4b32-bc89-5c19170c28c5 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:34.640748 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46198f-d355-4b32-bc89-5c19170c28c5 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:34.648769 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e80d2-cbd4-4dc2-bce9-869f44751400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:34.650949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e80d2-cbd4-4dc2-bce9-869f44751400 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:34.650949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e80d2-cbd4-4dc2-bce9-869f44751400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:34.650949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e80d2-cbd4-4dc2-bce9-869f44751400 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:34.652222 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e80d2-cbd4-4dc2-bce9-869f44751400 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:34.656595 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e80d2-cbd4-4dc2-bce9-869f44751400 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:34.689466 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-796afff6-3730-49ce-a48b-75a093f0dcfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a408f63dca4f4392bb176a50c0d72248'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:34.690143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-796afff6-3730-49ce-a48b-75a093f0dcfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a408f63dca4f4392bb176a50c0d72248'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:34.702496 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-796afff6-3730-49ce-a48b-75a093f0dcfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a408f63dca4f4392bb176a50c0d72248'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:34.709495 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-796afff6-3730-49ce-a48b-75a093f0dcfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a408f63dca4f4392bb176a50c0d72248'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:34.710245 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-796afff6-3730-49ce-a48b-75a093f0dcfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a408f63dca4f4392bb176a50c0d72248'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:34.847824 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1073/2154] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:54:34 2022] DELETE /identity/v3/users/a408f63dca4f4392bb176a50c0d72248 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:34.848870 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796afff6-3730-49ce-a48b-75a093f0dcfe admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:34.850338 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-796afff6-3730-49ce-a48b-75a093f0dcfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:34.859322 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1082/2155] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:54:34 2022] POST /identity/v3/projects => generated 383 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:54:34.860566 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e80d2-cbd4-4dc2-bce9-869f44751400 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:34.862663 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b58e80d2-cbd4-4dc2-bce9-869f44751400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:34.955859 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e80d2-cbd4-4dc2-bce9-869f44751400 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:34.961372 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74de8c-d31a-4006-8b1a-1addecdf232d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:34.961983 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74de8c-d31a-4006-8b1a-1addecdf232d admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:34.962385 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74de8c-d31a-4006-8b1a-1addecdf232d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:34.962791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74de8c-d31a-4006-8b1a-1addecdf232d admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:34.963623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b74de8c-d31a-4006-8b1a-1addecdf232d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:34.965599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b74de8c-d31a-4006-8b1a-1addecdf232d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:34.972554 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796afff6-3730-49ce-a48b-75a093f0dcfe admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:34.983524 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebdaf4d-f021-4e08-b2d8-dc62209ec1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:34.985455 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebdaf4d-f021-4e08-b2d8-dc62209ec1b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:34.990479 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebdaf4d-f021-4e08-b2d8-dc62209ec1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:34.990479 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebdaf4d-f021-4e08-b2d8-dc62209ec1b1 admin admin] PATH_INFO: `/v3/projects/2bf913b7df7e400e8b47f2fa67045763` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:34.990479 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebdaf4d-f021-4e08-b2d8-dc62209ec1b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bf913b7df7e400e8b47f2fa67045763)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:34.992420 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebdaf4d-f021-4e08-b2d8-dc62209ec1b1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:35.119992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1083/2156] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:54:34 2022] POST /identity/v3/users => generated 292 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:54:35.125913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74de8c-d31a-4006-8b1a-1addecdf232d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:35.141168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74de8c-d31a-4006-8b1a-1addecdf232d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:35.150925 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0539a5ed-c9e3-4416-8b77-23e71b0914f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:35.152000 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539a5ed-c9e3-4416-8b77-23e71b0914f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:35.152461 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539a5ed-c9e3-4416-8b77-23e71b0914f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:35.152947 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539a5ed-c9e3-4416-8b77-23e71b0914f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:35.154156 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0539a5ed-c9e3-4416-8b77-23e71b0914f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:35.169187 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0539a5ed-c9e3-4416-8b77-23e71b0914f9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:35.178519 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1084/2157] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:54:35.181256 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b7296-6c6b-4d42-9d68-4468c9649c23 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:35.181962 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b7296-6c6b-4d42-9d68-4468c9649c23 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:35.182409 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b7296-6c6b-4d42-9d68-4468c9649c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:35.192162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9ebdaf4d-f021-4e08-b2d8-dc62209ec1b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bf913b7df7e400e8b47f2fa67045763', 'request_id': 'req-9ebdaf4d-f021-4e08-b2d8-dc62209ec1b1'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:35.201490 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9ebdaf4d-f021-4e08-b2d8-dc62209ec1b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bf913b7df7e400e8b47f2fa67045763', 'request_id': 'req-9ebdaf4d-f021-4e08-b2d8-dc62209ec1b1'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:35.213740 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-122b7296-6c6b-4d42-9d68-4468c9649c23 None None] MFA Rules not processed for user `2ae70024479e488c90c6c1b955cb6471`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:35.215657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-122b7296-6c6b-4d42-9d68-4468c9649c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:35.287113 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1085/2158] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:54:35.288372 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b7296-6c6b-4d42-9d68-4468c9649c23 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:35.327294 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9ebdaf4d-f021-4e08-b2d8-dc62209ec1b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bf913b7df7e400e8b47f2fa67045763', 'request_id': 'req-9ebdaf4d-f021-4e08-b2d8-dc62209ec1b1'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:35.376439 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9ebdaf4d-f021-4e08-b2d8-dc62209ec1b1 admin admin] The token cache is being invalidate because project 2bf913b7df7e400e8b47f2fa67045763 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:54:35.377084 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9ebdaf4d-f021-4e08-b2d8-dc62209ec1b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:35.378175 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9ebdaf4d-f021-4e08-b2d8-dc62209ec1b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:35.380567 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1074/2159] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:54:34 2022] DELETE /identity/v3/projects/2bf913b7df7e400e8b47f2fa67045763 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:35.388579 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e83a9-4dc7-41de-b88e-1ef68ed47c4a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:35.389215 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e83a9-4dc7-41de-b88e-1ef68ed47c4a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:35.389689 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e83a9-4dc7-41de-b88e-1ef68ed47c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:35.405113 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b7296-6c6b-4d42-9d68-4468c9649c23 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:35.412598 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ef572-2bb6-4e4a-beed-33099189decd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:35.413329 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ef572-2bb6-4e4a-beed-33099189decd admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:35.413619 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ef572-2bb6-4e4a-beed-33099189decd admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:35.413854 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-218e83a9-4dc7-41de-b88e-1ef68ed47c4a None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:35.414070 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ef572-2bb6-4e4a-beed-33099189decd admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:35.414703 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454ef572-2bb6-4e4a-beed-33099189decd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:35.416558 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454ef572-2bb6-4e4a-beed-33099189decd admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:35.417019 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-218e83a9-4dc7-41de-b88e-1ef68ed47c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:35.444237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1086/2160] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:54:35.452647 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ef572-2bb6-4e4a-beed-33099189decd admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:35.458409 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-454ef572-2bb6-4e4a-beed-33099189decd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:35.458652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1075/2161] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:54:35.464066 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e83a9-4dc7-41de-b88e-1ef68ed47c4a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:35.465581 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-218e83a9-4dc7-41de-b88e-1ef68ed47c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:35.482064 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e83a9-4dc7-41de-b88e-1ef68ed47c4a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:35.496598 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4e25e-0a9d-4e99-90fd-dad694603e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:35.496598 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4e25e-0a9d-4e99-90fd-dad694603e1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:35.496598 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4e25e-0a9d-4e99-90fd-dad694603e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:35.497389 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ef572-2bb6-4e4a-beed-33099189decd admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:35.497558 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4e25e-0a9d-4e99-90fd-dad694603e1b admin admin] PATH_INFO: `/v3/projects/b4a8d632792f41d783924c9ff67873c7/users/e7d4cc581aa044cb8b10758f17f46ad6/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:35.503856 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361b1d8-96e0-44ad-bb0f-080b6b2010bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:35.504948 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361b1d8-96e0-44ad-bb0f-080b6b2010bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:35.510614 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361b1d8-96e0-44ad-bb0f-080b6b2010bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:35.515016 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361b1d8-96e0-44ad-bb0f-080b6b2010bd admin admin] PATH_INFO: `/v3/projects/8040bd92227543feac958a865ec46780` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:35.517537 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f361b1d8-96e0-44ad-bb0f-080b6b2010bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8040bd92227543feac958a865ec46780)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:35.521964 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f361b1d8-96e0-44ad-bb0f-080b6b2010bd admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:35.530118 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e4e25e-0a9d-4e99-90fd-dad694603e1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4a8d632792f41d783924c9ff67873c7, user_id=e7d4cc581aa044cb8b10758f17f46ad6, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:35.534470 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e4e25e-0a9d-4e99-90fd-dad694603e1b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:35.682630 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1076/2162] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:54:35 2022] PUT /identity/v3/projects/b4a8d632792f41d783924c9ff67873c7/users/e7d4cc581aa044cb8b10758f17f46ad6/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:35.684164 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2715fd-b64a-494a-9318-a2dde04f559b None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:35.684456 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2715fd-b64a-494a-9318-a2dde04f559b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:35.684733 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2715fd-b64a-494a-9318-a2dde04f559b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:35.717303 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f361b1d8-96e0-44ad-bb0f-080b6b2010bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8040bd92227543feac958a865ec46780', 'request_id': 'req-f361b1d8-96e0-44ad-bb0f-080b6b2010bd'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:35.732646 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f361b1d8-96e0-44ad-bb0f-080b6b2010bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8040bd92227543feac958a865ec46780', 'request_id': 'req-f361b1d8-96e0-44ad-bb0f-080b6b2010bd'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:35.757676 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-8e2715fd-b64a-494a-9318-a2dde04f559b None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:35.759451 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8e2715fd-b64a-494a-9318-a2dde04f559b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:35.870460 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1077/2163] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:54:35.873871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839cf56-6b7b-4ca6-ae36-c6a9f3d36bd8 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:35.874461 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839cf56-6b7b-4ca6-ae36-c6a9f3d36bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:35.875007 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839cf56-6b7b-4ca6-ae36-c6a9f3d36bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:35.876478 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f361b1d8-96e0-44ad-bb0f-080b6b2010bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8040bd92227543feac958a865ec46780', 'request_id': 'req-f361b1d8-96e0-44ad-bb0f-080b6b2010bd'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:35.919453 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c839cf56-6b7b-4ca6-ae36-c6a9f3d36bd8 None None] MFA Rules not processed for user `e7d4cc581aa044cb8b10758f17f46ad6`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:35.920611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c839cf56-6b7b-4ca6-ae36-c6a9f3d36bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:35.924780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f361b1d8-96e0-44ad-bb0f-080b6b2010bd admin admin] The token cache is being invalidate because project 8040bd92227543feac958a865ec46780 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:54:35.925198 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f361b1d8-96e0-44ad-bb0f-080b6b2010bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:35.928640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f361b1d8-96e0-44ad-bb0f-080b6b2010bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:35.938542 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1087/2164] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:54:35 2022] DELETE /identity/v3/projects/8040bd92227543feac958a865ec46780 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:35.943068 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361b1d8-96e0-44ad-bb0f-080b6b2010bd admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:35.945473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f361b1d8-96e0-44ad-bb0f-080b6b2010bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:35.976017 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361b1d8-96e0-44ad-bb0f-080b6b2010bd admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:35.986237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da5995-92f3-4c1e-84da-d80d13fa9ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:35.987306 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da5995-92f3-4c1e-84da-d80d13fa9ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:35.987857 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da5995-92f3-4c1e-84da-d80d13fa9ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:35.988672 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da5995-92f3-4c1e-84da-d80d13fa9ff0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:35.989832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da5995-92f3-4c1e-84da-d80d13fa9ff0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:54:35.990405 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da5995-92f3-4c1e-84da-d80d13fa9ff0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:35.993801 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da5995-92f3-4c1e-84da-d80d13fa9ff0 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:36.020518 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1088/2165] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:54:36.022153 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47531cbe-7d50-48e0-872f-948230c3f5b7 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:36.023372 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47531cbe-7d50-48e0-872f-948230c3f5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:36.023728 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47531cbe-7d50-48e0-872f-948230c3f5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:36.026944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1078/2166] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:54:36.028084 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839cf56-6b7b-4ca6-ae36-c6a9f3d36bd8 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:36.057009 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839cf56-6b7b-4ca6-ae36-c6a9f3d36bd8 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:36.059059 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-47531cbe-7d50-48e0-872f-948230c3f5b7 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:36.061170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-47531cbe-7d50-48e0-872f-948230c3f5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:36.071175 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086bbb53-63ad-480c-bb45-ee43fa278691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:36.072339 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bbb53-63ad-480c-bb45-ee43fa278691 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:36.072683 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bbb53-63ad-480c-bb45-ee43fa278691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:36.082112 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1089/2167] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:54:36.082314 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bbb53-63ad-480c-bb45-ee43fa278691 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:36.082314 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086bbb53-63ad-480c-bb45-ee43fa278691 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:36.082527 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47531cbe-7d50-48e0-872f-948230c3f5b7 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:36.082625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086bbb53-63ad-480c-bb45-ee43fa278691 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:36.090519 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-47531cbe-7d50-48e0-872f-948230c3f5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:36.113675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47531cbe-7d50-48e0-872f-948230c3f5b7 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:36.128424 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d942c46-4989-4fcc-9b9f-d5b1af7aacea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:36.133019 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d942c46-4989-4fcc-9b9f-d5b1af7aacea admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:36.133019 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d942c46-4989-4fcc-9b9f-d5b1af7aacea admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:36.133019 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d942c46-4989-4fcc-9b9f-d5b1af7aacea admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:36.133019 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d942c46-4989-4fcc-9b9f-d5b1af7aacea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:36.133019 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d942c46-4989-4fcc-9b9f-d5b1af7aacea admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:36.205809 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1079/2168] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:54:36 2022] POST /identity/v3/projects => generated 369 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:54:36.207101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e7ba4-fb57-4c9c-9e48-b9ddab6c4ee5 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:36.207390 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e7ba4-fb57-4c9c-9e48-b9ddab6c4ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:36.207707 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e7ba4-fb57-4c9c-9e48-b9ddab6c4ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:36.229063 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-5d0e7ba4-fb57-4c9c-9e48-b9ddab6c4ee5 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:36.230306 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5d0e7ba4-fb57-4c9c-9e48-b9ddab6c4ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:36.254624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1090/2169] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:54:36 2022] POST /identity/v3/projects => generated 387 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:54:36.254624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d942c46-4989-4fcc-9b9f-d5b1af7aacea admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:36.313976 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1080/2170] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:54:36.315077 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e7ba4-fb57-4c9c-9e48-b9ddab6c4ee5 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:36.322935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:36.327125 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e7ba4-fb57-4c9c-9e48-b9ddab6c4ee5 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:36.346641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d942c46-4989-4fcc-9b9f-d5b1af7aacea admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:36.353285 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f9b64-b7a7-4031-9774-9cda216a27e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:36.354230 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f9b64-b7a7-4031-9774-9cda216a27e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:36.354631 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f9b64-b7a7-4031-9774-9cda216a27e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:36.357232 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f9b64-b7a7-4031-9774-9cda216a27e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:36.357232 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f9b64-b7a7-4031-9774-9cda216a27e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:36.358228 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f9b64-b7a7-4031-9774-9cda216a27e1 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:36.393422 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15b175-ca27-4ba3-8e5f-4d216bc10f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:36.394055 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15b175-ca27-4ba3-8e5f-4d216bc10f2e admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:36.394342 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15b175-ca27-4ba3-8e5f-4d216bc10f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:36.394625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15b175-ca27-4ba3-8e5f-4d216bc10f2e admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:36.395294 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c15b175-ca27-4ba3-8e5f-4d216bc10f2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:36.397373 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c15b175-ca27-4ba3-8e5f-4d216bc10f2e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:36.459316 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:36.474593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1091/2171] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:54:36 2022] POST /identity/v3/users => generated 285 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:54:36.474593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f9b64-b7a7-4031-9774-9cda216a27e1 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:36.492374 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f9b64-b7a7-4031-9774-9cda216a27e1 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:36.500178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef25cb-30ba-4476-86f7-f7b7753a5521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:36.501083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef25cb-30ba-4476-86f7-f7b7753a5521 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:36.501506 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef25cb-30ba-4476-86f7-f7b7753a5521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:36.501953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef25cb-30ba-4476-86f7-f7b7753a5521 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:36.503052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef25cb-30ba-4476-86f7-f7b7753a5521 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:54:36.503682 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef25cb-30ba-4476-86f7-f7b7753a5521 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:36.506237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef25cb-30ba-4476-86f7-f7b7753a5521 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:36.523516 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1081/2172] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:54:36 2022] POST /identity/v3/users => generated 294 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:54:36.529784 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1092/2173] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:54:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:54:36.529936 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15b175-ca27-4ba3-8e5f-4d216bc10f2e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:36.530033 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef25cb-30ba-4476-86f7-f7b7753a5521 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:36.539523 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef25cb-30ba-4476-86f7-f7b7753a5521 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:36.540741 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15b175-ca27-4ba3-8e5f-4d216bc10f2e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:36.548532 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781485fc-fdec-47a4-ab95-4c53b8a7e167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:36.549648 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc5a9b-d7af-44d7-96a5-64846db2a6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:36.549811 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781485fc-fdec-47a4-ab95-4c53b8a7e167 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:36.550108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781485fc-fdec-47a4-ab95-4c53b8a7e167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:36.550296 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc5a9b-d7af-44d7-96a5-64846db2a6a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:36.550584 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781485fc-fdec-47a4-ab95-4c53b8a7e167 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:36.550771 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc5a9b-d7af-44d7-96a5-64846db2a6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:36.550933 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc5a9b-d7af-44d7-96a5-64846db2a6a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:36.551634 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781485fc-fdec-47a4-ab95-4c53b8a7e167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:36.551839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc5a9b-d7af-44d7-96a5-64846db2a6a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:36.553270 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc5a9b-d7af-44d7-96a5-64846db2a6a6 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:36.554072 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781485fc-fdec-47a4-ab95-4c53b8a7e167 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:36.566426 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1093/2174] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:54:36.566426 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc5a9b-d7af-44d7-96a5-64846db2a6a6 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:36.570030 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1082/2175] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:36 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:54:36.573489 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781485fc-fdec-47a4-ab95-4c53b8a7e167 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:36.578672 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc5a9b-d7af-44d7-96a5-64846db2a6a6 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:36.584222 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffde6d4-81ec-4054-9e86-691bda4fdca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:36.584974 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffde6d4-81ec-4054-9e86-691bda4fdca3 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:36.585273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffde6d4-81ec-4054-9e86-691bda4fdca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:36.585528 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffde6d4-81ec-4054-9e86-691bda4fdca3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:36.586202 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffde6d4-81ec-4054-9e86-691bda4fdca3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:36.587495 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781485fc-fdec-47a4-ab95-4c53b8a7e167 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:36.588165 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffde6d4-81ec-4054-9e86-691bda4fdca3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:36.593259 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6619bf-bcd8-40ca-803e-e67efd74089f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:36.594129 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6619bf-bcd8-40ca-803e-e67efd74089f admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:36.594533 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6619bf-bcd8-40ca-803e-e67efd74089f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:36.595161 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6619bf-bcd8-40ca-803e-e67efd74089f admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:36.596310 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6619bf-bcd8-40ca-803e-e67efd74089f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:36.597981 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6619bf-bcd8-40ca-803e-e67efd74089f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:36.610364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1083/2176] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:54:36.611279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6619bf-bcd8-40ca-803e-e67efd74089f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:36.621343 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6619bf-bcd8-40ca-803e-e67efd74089f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:36.626815 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3acb065-f57c-4fe0-81f4-1964b76b7ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:36.628723 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acb065-f57c-4fe0-81f4-1964b76b7ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:36.628723 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acb065-f57c-4fe0-81f4-1964b76b7ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:36.628723 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acb065-f57c-4fe0-81f4-1964b76b7ee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:36.628723 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3acb065-f57c-4fe0-81f4-1964b76b7ee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:36.630303 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3acb065-f57c-4fe0-81f4-1964b76b7ee2 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:36.641777 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1084/2177] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:54:36.642623 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3acb065-f57c-4fe0-81f4-1964b76b7ee2 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:36.652868 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3acb065-f57c-4fe0-81f4-1964b76b7ee2 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:36.657815 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1a9cf-cc06-4379-8038-7f2d312782ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:36.658483 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1a9cf-cc06-4379-8038-7f2d312782ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:36.658768 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1a9cf-cc06-4379-8038-7f2d312782ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:36.659047 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1a9cf-cc06-4379-8038-7f2d312782ff admin admin] PATH_INFO: `/v3/projects/b422aa7594fd4822be9223119ff1ae77/users/348a0cdc31034d518aff9257eda00c9b/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:36.674755 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b1a9cf-cc06-4379-8038-7f2d312782ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=b422aa7594fd4822be9223119ff1ae77, user_id=348a0cdc31034d518aff9257eda00c9b, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:36.677562 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b1a9cf-cc06-4379-8038-7f2d312782ff admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:36.727499 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1094/2178] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:54:36 2022] POST /identity/v3/projects => generated 367 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:54:36.731537 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffde6d4-81ec-4054-9e86-691bda4fdca3 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:36.814060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffde6d4-81ec-4054-9e86-691bda4fdca3 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:36.816131 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1085/2179] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:54:36 2022] PUT /identity/v3/projects/b422aa7594fd4822be9223119ff1ae77/users/348a0cdc31034d518aff9257eda00c9b/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:36.817600 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1a9cf-cc06-4379-8038-7f2d312782ff admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:36.923288 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b8e0b-79fb-45ff-9277-c841840566c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:36.924421 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b8e0b-79fb-45ff-9277-c841840566c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:36.924865 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b8e0b-79fb-45ff-9277-c841840566c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:36.925221 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b8e0b-79fb-45ff-9277-c841840566c1 admin admin] PATH_INFO: `/v3/projects/8d9966e52d2e4bc8a0f5659f2f494104/users/b5dfeaec44c84504901041aefb801f93/roles/e033961a9363456a8510c34414dd839e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:36.932752 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1a9cf-cc06-4379-8038-7f2d312782ff admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:36.938577 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86ef43-5e69-4b53-831f-3eac44d663aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:36.939229 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86ef43-5e69-4b53-831f-3eac44d663aa admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:36.939539 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86ef43-5e69-4b53-831f-3eac44d663aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:36.939812 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86ef43-5e69-4b53-831f-3eac44d663aa admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:36.940477 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec86ef43-5e69-4b53-831f-3eac44d663aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:36.942406 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec86ef43-5e69-4b53-831f-3eac44d663aa admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:36.950085 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b8e0b-79fb-45ff-9277-c841840566c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d9966e52d2e4bc8a0f5659f2f494104, user_id=b5dfeaec44c84504901041aefb801f93, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:36.954912 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b8e0b-79fb-45ff-9277-c841840566c1 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:37.141519 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1095/2180] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:54:36 2022] PUT /identity/v3/projects/8d9966e52d2e4bc8a0f5659f2f494104/users/b5dfeaec44c84504901041aefb801f93/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:37.143278 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc6c64-4c52-4962-baf5-45dff0a076bc None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:37.154086 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1086/2181] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:54:36 2022] POST /identity/v3/users => generated 284 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:54:37.154290 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc6c64-4c52-4962-baf5-45dff0a076bc None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:37.154290 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc6c64-4c52-4962-baf5-45dff0a076bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:37.163544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86ef43-5e69-4b53-831f-3eac44d663aa admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:37.222802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f8dc6c64-4c52-4962-baf5-45dff0a076bc None None] MFA Rules not processed for user `348a0cdc31034d518aff9257eda00c9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:37.224096 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f8dc6c64-4c52-4962-baf5-45dff0a076bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:37.225606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86ef43-5e69-4b53-831f-3eac44d663aa admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:37.340302 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae6d9d-45b3-432d-be23-b91dbd7fc2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:37.340961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae6d9d-45b3-432d-be23-b91dbd7fc2ab admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:37.341265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae6d9d-45b3-432d-be23-b91dbd7fc2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:37.341606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae6d9d-45b3-432d-be23-b91dbd7fc2ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:37.342250 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae6d9d-45b3-432d-be23-b91dbd7fc2ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:37.344464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae6d9d-45b3-432d-be23-b91dbd7fc2ab admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:37.344713 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1096/2182] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:54:37.346348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe64cce-bc0a-4ede-8c79-14b1f6621f82 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:37.346825 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe64cce-bc0a-4ede-8c79-14b1f6621f82 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:37.347378 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe64cce-bc0a-4ede-8c79-14b1f6621f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:37.359620 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1087/2183] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:54:37.363274 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae6d9d-45b3-432d-be23-b91dbd7fc2ab admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:37.374047 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae6d9d-45b3-432d-be23-b91dbd7fc2ab admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:37.379235 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c8532-7b5b-4e21-93d1-5171d3b2891e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:37.379923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c8532-7b5b-4e21-93d1-5171d3b2891e admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:37.380304 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c8532-7b5b-4e21-93d1-5171d3b2891e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:37.380482 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c8532-7b5b-4e21-93d1-5171d3b2891e admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:37.380635 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3fe64cce-bc0a-4ede-8c79-14b1f6621f82 None None] MFA Rules not processed for user `b5dfeaec44c84504901041aefb801f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:37.381145 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c8532-7b5b-4e21-93d1-5171d3b2891e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:37.382082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3fe64cce-bc0a-4ede-8c79-14b1f6621f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:37.383105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c8532-7b5b-4e21-93d1-5171d3b2891e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:37.460965 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1097/2184] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:37 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:54:37.467984 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe64cce-bc0a-4ede-8c79-14b1f6621f82 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:37.482580 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe64cce-bc0a-4ede-8c79-14b1f6621f82 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:37.489100 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc96dc3-6865-48a2-8051-e8fcbd943888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:37.489750 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc96dc3-6865-48a2-8051-e8fcbd943888 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:37.490034 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc96dc3-6865-48a2-8051-e8fcbd943888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:37.490314 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc96dc3-6865-48a2-8051-e8fcbd943888 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:37.491041 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc96dc3-6865-48a2-8051-e8fcbd943888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:37.492892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc96dc3-6865-48a2-8051-e8fcbd943888 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:37.505009 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1098/2185] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:54:37.505877 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc96dc3-6865-48a2-8051-e8fcbd943888 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:37.507774 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-efc96dc3-6865-48a2-8051-e8fcbd943888 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:37.588609 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1088/2186] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:54:37 2022] POST /identity/v3/projects => generated 385 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:54:37.589327 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c8532-7b5b-4e21-93d1-5171d3b2891e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:37.627566 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc96dc3-6865-48a2-8051-e8fcbd943888 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:37.682375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c8532-7b5b-4e21-93d1-5171d3b2891e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:37.689929 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07603e7-a56c-4b95-92c9-c7e32c872a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:37.691049 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07603e7-a56c-4b95-92c9-c7e32c872a1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:37.691543 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07603e7-a56c-4b95-92c9-c7e32c872a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:37.692434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07603e7-a56c-4b95-92c9-c7e32c872a1c admin admin] PATH_INFO: `/v3/projects/03cf2de1c32546ef8053f2e1de3a7cba/users/6c28ba3fae544ba58e066bee526d83a3/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:37.713329 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d77d5-06c4-4965-b9dc-9fc0e9b4dacf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62adb1f5e9654927847e1717630ff6b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:37.713963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d77d5-06c4-4965-b9dc-9fc0e9b4dacf service glance] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:37.714252 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d77d5-06c4-4965-b9dc-9fc0e9b4dacf service glance] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:37.714535 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d77d5-06c4-4965-b9dc-9fc0e9b4dacf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:37.732405 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07603e7-a56c-4b95-92c9-c7e32c872a1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=03cf2de1c32546ef8053f2e1de3a7cba, user_id=6c28ba3fae544ba58e066bee526d83a3, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:37.735769 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07603e7-a56c-4b95-92c9-c7e32c872a1c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:37.786728 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d77d5-06c4-4965-b9dc-9fc0e9b4dacf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:37.788223 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d77d5-06c4-4965-b9dc-9fc0e9b4dacf service glance] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:37.800197 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1099/2187] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:54:37 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:54:37.808606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d77d5-06c4-4965-b9dc-9fc0e9b4dacf service glance] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:37.822795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d77d5-06c4-4965-b9dc-9fc0e9b4dacf service glance] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:37.831485 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e65596a-4c2a-460a-b05b-18d4ca098370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:37.832181 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e65596a-4c2a-460a-b05b-18d4ca098370 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:37.832473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e65596a-4c2a-460a-b05b-18d4ca098370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:37.832754 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e65596a-4c2a-460a-b05b-18d4ca098370 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:37.833429 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e65596a-4c2a-460a-b05b-18d4ca098370 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:37.835658 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e65596a-4c2a-460a-b05b-18d4ca098370 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:37.844380 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1089/2188] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:54:37 2022] PUT /identity/v3/projects/03cf2de1c32546ef8053f2e1de3a7cba/users/6c28ba3fae544ba58e066bee526d83a3/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:37.862437 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955a374-4af2-4db7-bf47-6111bc9a24a3 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:37.862752 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955a374-4af2-4db7-bf47-6111bc9a24a3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:37.863036 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955a374-4af2-4db7-bf47-6111bc9a24a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:37.896010 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-f955a374-4af2-4db7-bf47-6111bc9a24a3 None None] MFA Rules not processed for user `6c28ba3fae544ba58e066bee526d83a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:37.897922 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f955a374-4af2-4db7-bf47-6111bc9a24a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:37.941573 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1100/2189] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:54:37 2022] POST /identity/v3/users => generated 299 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:54:37.947107 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e65596a-4c2a-460a-b05b-18d4ca098370 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:37.977498 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e65596a-4c2a-460a-b05b-18d4ca098370 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:38.055507 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1090/2190] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:37 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:54:38.057136 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955a374-4af2-4db7-bf47-6111bc9a24a3 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:38.148094 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc4193e-7cb5-4b65-8b69-f100f52fd2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:38.149418 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4193e-7cb5-4b65-8b69-f100f52fd2e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:38.149923 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4193e-7cb5-4b65-8b69-f100f52fd2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:38.150446 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4193e-7cb5-4b65-8b69-f100f52fd2e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:38.151531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc4193e-7cb5-4b65-8b69-f100f52fd2e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:38.153389 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc4193e-7cb5-4b65-8b69-f100f52fd2e7 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:38.165848 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1101/2191] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:54:38.172425 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc4193e-7cb5-4b65-8b69-f100f52fd2e7 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:38.186265 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc4193e-7cb5-4b65-8b69-f100f52fd2e7 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:38.192782 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa23b7a-916e-4dc0-84b6-11c65a391a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:38.193998 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa23b7a-916e-4dc0-84b6-11c65a391a0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:38.196040 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa23b7a-916e-4dc0-84b6-11c65a391a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:38.196538 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa23b7a-916e-4dc0-84b6-11c65a391a0f admin admin] PATH_INFO: `/v3/projects/57651e1b425849dda5c655426b9462dd/users/4a2e27fc22d746ac8f8ff348219bc486/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:38.221127 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa23b7a-916e-4dc0-84b6-11c65a391a0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=57651e1b425849dda5c655426b9462dd, user_id=4a2e27fc22d746ac8f8ff348219bc486, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:38.225384 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa23b7a-916e-4dc0-84b6-11c65a391a0f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:38.283728 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955a374-4af2-4db7-bf47-6111bc9a24a3 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:38.302929 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f8764c-da8a-413e-9c5c-f6a14d5bb77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:38.304014 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f8764c-da8a-413e-9c5c-f6a14d5bb77f admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:38.304466 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f8764c-da8a-413e-9c5c-f6a14d5bb77f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:38.304919 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f8764c-da8a-413e-9c5c-f6a14d5bb77f admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:38.306096 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f8764c-da8a-413e-9c5c-f6a14d5bb77f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:38.312399 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f8764c-da8a-413e-9c5c-f6a14d5bb77f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:38.334760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1102/2192] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:54:38 2022] PUT /identity/v3/projects/57651e1b425849dda5c655426b9462dd/users/4a2e27fc22d746ac8f8ff348219bc486/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:38.340533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa23b7a-916e-4dc0-84b6-11c65a391a0f admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:38.462361 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa23b7a-916e-4dc0-84b6-11c65a391a0f admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:38.467537 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1091/2193] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:54:38 2022] POST /identity/v3/projects => generated 369 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:54:38.468055 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa03cd-13c9-4a53-8bb9-838e40be9b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:38.469416 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa03cd-13c9-4a53-8bb9-838e40be9b84 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:38.469621 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa03cd-13c9-4a53-8bb9-838e40be9b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:38.469870 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa03cd-13c9-4a53-8bb9-838e40be9b84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:38.470733 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befa03cd-13c9-4a53-8bb9-838e40be9b84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:38.472434 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befa03cd-13c9-4a53-8bb9-838e40be9b84 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:38.474177 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f8764c-da8a-413e-9c5c-f6a14d5bb77f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:38.484613 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1103/2194] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:54:38.489913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa03cd-13c9-4a53-8bb9-838e40be9b84 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:38.503018 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa03cd-13c9-4a53-8bb9-838e40be9b84 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:38.582970 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f98ad-eead-4a6f-b5ca-3ed63faacbb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:38.583992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f98ad-eead-4a6f-b5ca-3ed63faacbb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:38.584584 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f98ad-eead-4a6f-b5ca-3ed63faacbb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:38.585039 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f98ad-eead-4a6f-b5ca-3ed63faacbb1 admin admin] PATH_INFO: `/v3/projects/57651e1b425849dda5c655426b9462dd/users/4a2e27fc22d746ac8f8ff348219bc486/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:38.587306 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f98ad-eead-4a6f-b5ca-3ed63faacbb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57651e1b425849dda5c655426b9462dd, user_id=4a2e27fc22d746ac8f8ff348219bc486, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:38.591085 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f98ad-eead-4a6f-b5ca-3ed63faacbb1 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:38.605035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1104/2195] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:54:38 2022] PUT /identity/v3/projects/57651e1b425849dda5c655426b9462dd/users/4a2e27fc22d746ac8f8ff348219bc486/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:54:38.611709 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f8764c-da8a-413e-9c5c-f6a14d5bb77f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:38.621904 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c9f6f-a0a5-4dac-86c3-3e5918c2e7e6 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:38.622292 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c9f6f-a0a5-4dac-86c3-3e5918c2e7e6 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:38.622469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c9f6f-a0a5-4dac-86c3-3e5918c2e7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:38.670236 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a42c9f6f-a0a5-4dac-86c3-3e5918c2e7e6 None None] MFA Rules not processed for user `4a2e27fc22d746ac8f8ff348219bc486`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:38.672817 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a42c9f6f-a0a5-4dac-86c3-3e5918c2e7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:38.780264 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b9952-7716-4955-aec3-ca8f52b13027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:38.781586 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b9952-7716-4955-aec3-ca8f52b13027 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:38.782069 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b9952-7716-4955-aec3-ca8f52b13027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:38.782551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b9952-7716-4955-aec3-ca8f52b13027 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:38.786667 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1105/2196] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:38 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:54:38.791772 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b9952-7716-4955-aec3-ca8f52b13027 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:38.812889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c9f6f-a0a5-4dac-86c3-3e5918c2e7e6 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:38.813633 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b9952-7716-4955-aec3-ca8f52b13027 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:38.814446 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a42c9f6f-a0a5-4dac-86c3-3e5918c2e7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:38.907034 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c9f6f-a0a5-4dac-86c3-3e5918c2e7e6 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:38.911279 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd7f8e-a458-48c4-bf4f-4bf26d34fd5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:38.912070 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd7f8e-a458-48c4-bf4f-4bf26d34fd5b service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:38.912359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd7f8e-a458-48c4-bf4f-4bf26d34fd5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:38.912637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd7f8e-a458-48c4-bf4f-4bf26d34fd5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:38.928299 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd7f8e-a458-48c4-bf4f-4bf26d34fd5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:38.928299 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd7f8e-a458-48c4-bf4f-4bf26d34fd5b service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:38.940010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1106/2197] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:54:38 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:54:38.973723 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:38.981717 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1092/2198] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:54:38 2022] POST /identity/v3/users => generated 291 bytes in 508 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:54:38.985830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd7f8e-a458-48c4-bf4f-4bf26d34fd5b service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:38.996343 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd7f8e-a458-48c4-bf4f-4bf26d34fd5b service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:39.017627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad160c-5f2c-4dd6-81dd-eb7e92d68658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:39.018606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad160c-5f2c-4dd6-81dd-eb7e92d68658 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:39.019702 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad160c-5f2c-4dd6-81dd-eb7e92d68658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:39.020000 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad160c-5f2c-4dd6-81dd-eb7e92d68658 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:39.020979 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad160c-5f2c-4dd6-81dd-eb7e92d68658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:39.023105 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad160c-5f2c-4dd6-81dd-eb7e92d68658 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:39.059890 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1107/2199] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:54:39.065491 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b9952-7716-4955-aec3-ca8f52b13027 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:39.097401 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b9952-7716-4955-aec3-ca8f52b13027 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:39.101053 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:39.121888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ead67-3193-4798-b3e0-2d19c0a99d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:39.122498 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ead67-3193-4798-b3e0-2d19c0a99d44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:39.122805 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ead67-3193-4798-b3e0-2d19c0a99d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:39.123087 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ead67-3193-4798-b3e0-2d19c0a99d44 admin admin] PATH_INFO: `/v3/projects/af59d2d5839543e6aa2f0eedc5db3f54/users/bc7f9843b19e4ea599916d65ba765dd6/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:39.154841 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48ead67-3193-4798-b3e0-2d19c0a99d44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af59d2d5839543e6aa2f0eedc5db3f54, user_id=bc7f9843b19e4ea599916d65ba765dd6, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:39.159255 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48ead67-3193-4798-b3e0-2d19c0a99d44 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:39.324646 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31349e-c44e-464b-af1e-2e1c32e48970 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:39.325323 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31349e-c44e-464b-af1e-2e1c32e48970 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:39.325838 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31349e-c44e-464b-af1e-2e1c32e48970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:39.336215 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1093/2200] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:54:39 2022] PUT /identity/v3/projects/af59d2d5839543e6aa2f0eedc5db3f54/users/bc7f9843b19e4ea599916d65ba765dd6/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:39.351802 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ead67-3193-4798-b3e0-2d19c0a99d44 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:39.368327 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-1c31349e-c44e-464b-af1e-2e1c32e48970 None None] MFA Rules not processed for user `19c0408b4321495eabd4738bbb8b711b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:39.372411 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1c31349e-c44e-464b-af1e-2e1c32e48970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:39.496650 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ead67-3193-4798-b3e0-2d19c0a99d44 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:39.502382 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bdab61-9444-4a04-b14d-95d42ccb7242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:39.503227 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bdab61-9444-4a04-b14d-95d42ccb7242 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:39.503662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bdab61-9444-4a04-b14d-95d42ccb7242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:39.504042 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bdab61-9444-4a04-b14d-95d42ccb7242 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:39.504855 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bdab61-9444-4a04-b14d-95d42ccb7242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:39.507055 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bdab61-9444-4a04-b14d-95d42ccb7242 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:39.513052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1108/2201] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:54:39.542188 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1094/2202] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:39 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:54:39.554672 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31349e-c44e-464b-af1e-2e1c32e48970 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:39.590949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31349e-c44e-464b-af1e-2e1c32e48970 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:39.611368 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d0361-4a15-4ae6-9ec5-c22a9ea3a590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:39.612268 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d0361-4a15-4ae6-9ec5-c22a9ea3a590 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:39.612699 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d0361-4a15-4ae6-9ec5-c22a9ea3a590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:39.612988 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d0361-4a15-4ae6-9ec5-c22a9ea3a590 admin admin] PATH_INFO: `/v3/projects/af59d2d5839543e6aa2f0eedc5db3f54/users/bc7f9843b19e4ea599916d65ba765dd6/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:39.616340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d0361-4a15-4ae6-9ec5-c22a9ea3a590 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af59d2d5839543e6aa2f0eedc5db3f54, user_id=bc7f9843b19e4ea599916d65ba765dd6, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:39.619225 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d0361-4a15-4ae6-9ec5-c22a9ea3a590 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:39.667908 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1109/2203] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:54:39 2022] PUT /identity/v3/projects/af59d2d5839543e6aa2f0eedc5db3f54/users/bc7f9843b19e4ea599916d65ba765dd6/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:39.684853 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deda1ca-7566-4e1d-ac65-5f3e476c50a8 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:39.685631 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deda1ca-7566-4e1d-ac65-5f3e476c50a8 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:39.686701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deda1ca-7566-4e1d-ac65-5f3e476c50a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:39.742467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-5deda1ca-7566-4e1d-ac65-5f3e476c50a8 None None] MFA Rules not processed for user `bc7f9843b19e4ea599916d65ba765dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:39.744379 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5deda1ca-7566-4e1d-ac65-5f3e476c50a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:39.871387 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1095/2204] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:39 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:54:39.882962 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d0361-4a15-4ae6-9ec5-c22a9ea3a590 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:39.959473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d0361-4a15-4ae6-9ec5-c22a9ea3a590 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:39.963650 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb40f1-cbf7-4a87-abd4-945aa5cc5917 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:39.964743 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb40f1-cbf7-4a87-abd4-945aa5cc5917 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:39.965100 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb40f1-cbf7-4a87-abd4-945aa5cc5917 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:39.965496 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb40f1-cbf7-4a87-abd4-945aa5cc5917 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:39.990231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb40f1-cbf7-4a87-abd4-945aa5cc5917 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:39.992113 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb40f1-cbf7-4a87-abd4-945aa5cc5917 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:40.006346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1110/2205] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:54:39 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:54:40.814705 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deda1ca-7566-4e1d-ac65-5f3e476c50a8 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:40.816838 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5deda1ca-7566-4e1d-ac65-5f3e476c50a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:40.842704 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deda1ca-7566-4e1d-ac65-5f3e476c50a8 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:40.851770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4210e-2b0d-4ce0-9786-ebbfb68881db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:40.852624 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4210e-2b0d-4ce0-9786-ebbfb68881db service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:40.853136 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4210e-2b0d-4ce0-9786-ebbfb68881db service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:40.853605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4210e-2b0d-4ce0-9786-ebbfb68881db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:40.947996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa4210e-2b0d-4ce0-9786-ebbfb68881db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:40.949872 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa4210e-2b0d-4ce0-9786-ebbfb68881db service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:40.967068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1096/2206] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:54:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:54:41.351973 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:41.524038 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:43.106111 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:43.277467 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:44.014748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:44.197880 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:44.400359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb40f1-cbf7-4a87-abd4-945aa5cc5917 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:44.516825 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb40f1-cbf7-4a87-abd4-945aa5cc5917 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:44.532294 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c374da0-187a-472f-bafa-0d8f640b032f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:44.532294 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c374da0-187a-472f-bafa-0d8f640b032f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:44.532294 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c374da0-187a-472f-bafa-0d8f640b032f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:44.532899 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c374da0-187a-472f-bafa-0d8f640b032f admin admin] PATH_INFO: `/v3/users/6c28ba3fae544ba58e066bee526d83a3` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:44.534393 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c374da0-187a-472f-bafa-0d8f640b032f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c28ba3fae544ba58e066bee526d83a3)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:44.536958 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c374da0-187a-472f-bafa-0d8f640b032f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:44.776184 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6c374da0-187a-472f-bafa-0d8f640b032f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c28ba3fae544ba58e066bee526d83a3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:44.776852 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6c374da0-187a-472f-bafa-0d8f640b032f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c28ba3fae544ba58e066bee526d83a3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:44.788604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6c374da0-187a-472f-bafa-0d8f640b032f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c28ba3fae544ba58e066bee526d83a3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:44.796251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6c374da0-187a-472f-bafa-0d8f640b032f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c28ba3fae544ba58e066bee526d83a3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:44.797539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6c374da0-187a-472f-bafa-0d8f640b032f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c28ba3fae544ba58e066bee526d83a3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:44.890618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1111/2207] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:54:44 2022] DELETE /identity/v3/users/6c28ba3fae544ba58e066bee526d83a3 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:44.895028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195db3fb-a17a-452b-9151-9b72a37db5d8 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:44.895529 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195db3fb-a17a-452b-9151-9b72a37db5d8 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:44.895624 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195db3fb-a17a-452b-9151-9b72a37db5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:44.915931 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-195db3fb-a17a-452b-9151-9b72a37db5d8 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:44.917212 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-195db3fb-a17a-452b-9151-9b72a37db5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:44.979449 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1097/2208] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:54:44.989191 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c374da0-187a-472f-bafa-0d8f640b032f admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:44.990588 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6c374da0-187a-472f-bafa-0d8f640b032f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:45.026736 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:45.057806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c374da0-187a-472f-bafa-0d8f640b032f admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:45.061525 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a062921-547f-44da-9400-b060b7aa97ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:45.062149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a062921-547f-44da-9400-b060b7aa97ca service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:45.062433 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a062921-547f-44da-9400-b060b7aa97ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:45.062711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a062921-547f-44da-9400-b060b7aa97ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:45.074697 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a062921-547f-44da-9400-b060b7aa97ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:45.076179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a062921-547f-44da-9400-b060b7aa97ca service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:45.086098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1112/2209] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:54:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:54:45.158105 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:45.689595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195db3fb-a17a-452b-9151-9b72a37db5d8 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:45.691673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-195db3fb-a17a-452b-9151-9b72a37db5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:45.719294 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195db3fb-a17a-452b-9151-9b72a37db5d8 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:45.746433 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc2239-ad66-4440-96de-0275ddca4c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:45.747378 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc2239-ad66-4440-96de-0275ddca4c20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:45.748424 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc2239-ad66-4440-96de-0275ddca4c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:45.748819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc2239-ad66-4440-96de-0275ddca4c20 admin admin] PATH_INFO: `/v3/projects/03cf2de1c32546ef8053f2e1de3a7cba` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:45.750866 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc2239-ad66-4440-96de-0275ddca4c20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03cf2de1c32546ef8053f2e1de3a7cba)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:45.752960 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc2239-ad66-4440-96de-0275ddca4c20 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:45.876314 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-61cc2239-ad66-4440-96de-0275ddca4c20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03cf2de1c32546ef8053f2e1de3a7cba', 'request_id': 'req-61cc2239-ad66-4440-96de-0275ddca4c20'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:45.885754 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-61cc2239-ad66-4440-96de-0275ddca4c20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03cf2de1c32546ef8053f2e1de3a7cba', 'request_id': 'req-61cc2239-ad66-4440-96de-0275ddca4c20'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:45.968897 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-61cc2239-ad66-4440-96de-0275ddca4c20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03cf2de1c32546ef8053f2e1de3a7cba', 'request_id': 'req-61cc2239-ad66-4440-96de-0275ddca4c20'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:45.977692 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:45.981459 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-61cc2239-ad66-4440-96de-0275ddca4c20 admin admin] The token cache is being invalidate because project 03cf2de1c32546ef8053f2e1de3a7cba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:54:45.981749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-61cc2239-ad66-4440-96de-0275ddca4c20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:45.982467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-61cc2239-ad66-4440-96de-0275ddca4c20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:45.984893 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1098/2210] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:54:45 2022] DELETE /identity/v3/projects/03cf2de1c32546ef8053f2e1de3a7cba => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:45.989130 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a062921-547f-44da-9400-b060b7aa97ca service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:45.998804 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a062921-547f-44da-9400-b060b7aa97ca service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:46.031491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:46.059763 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05931005-5011-4032-9924-630063b7573f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:46.060404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05931005-5011-4032-9924-630063b7573f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:46.060687 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05931005-5011-4032-9924-630063b7573f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:46.061008 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05931005-5011-4032-9924-630063b7573f admin admin] PATH_INFO: `/v3/users/bc7f9843b19e4ea599916d65ba765dd6` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:46.062855 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05931005-5011-4032-9924-630063b7573f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc7f9843b19e4ea599916d65ba765dd6)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:46.064955 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05931005-5011-4032-9924-630063b7573f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:46.382903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-05931005-5011-4032-9924-630063b7573f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc7f9843b19e4ea599916d65ba765dd6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:46.384183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-05931005-5011-4032-9924-630063b7573f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc7f9843b19e4ea599916d65ba765dd6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:46.408851 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-05931005-5011-4032-9924-630063b7573f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc7f9843b19e4ea599916d65ba765dd6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:46.416689 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-05931005-5011-4032-9924-630063b7573f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc7f9843b19e4ea599916d65ba765dd6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:46.423362 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-05931005-5011-4032-9924-630063b7573f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc7f9843b19e4ea599916d65ba765dd6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:46.541829 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1113/2211] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:54:45 2022] DELETE /identity/v3/users/bc7f9843b19e4ea599916d65ba765dd6 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:47.183026 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc2239-ad66-4440-96de-0275ddca4c20 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:47.184915 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-61cc2239-ad66-4440-96de-0275ddca4c20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:47.214702 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc2239-ad66-4440-96de-0275ddca4c20 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:47.286119 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae778e7-d3db-478a-8bce-382527eab676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:47.286911 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae778e7-d3db-478a-8bce-382527eab676 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:47.287267 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae778e7-d3db-478a-8bce-382527eab676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:47.287650 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae778e7-d3db-478a-8bce-382527eab676 admin admin] PATH_INFO: `/v3/projects/af59d2d5839543e6aa2f0eedc5db3f54` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:47.289022 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae778e7-d3db-478a-8bce-382527eab676 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af59d2d5839543e6aa2f0eedc5db3f54)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:47.291056 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae778e7-d3db-478a-8bce-382527eab676 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:47.450974 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2ae778e7-d3db-478a-8bce-382527eab676 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af59d2d5839543e6aa2f0eedc5db3f54', 'request_id': 'req-2ae778e7-d3db-478a-8bce-382527eab676'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:47.458171 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2ae778e7-d3db-478a-8bce-382527eab676 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af59d2d5839543e6aa2f0eedc5db3f54', 'request_id': 'req-2ae778e7-d3db-478a-8bce-382527eab676'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:47.483190 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:47.619895 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2ae778e7-d3db-478a-8bce-382527eab676 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af59d2d5839543e6aa2f0eedc5db3f54', 'request_id': 'req-2ae778e7-d3db-478a-8bce-382527eab676'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:47.634285 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2ae778e7-d3db-478a-8bce-382527eab676 admin admin] The token cache is being invalidate because project af59d2d5839543e6aa2f0eedc5db3f54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:54:47.634581 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2ae778e7-d3db-478a-8bce-382527eab676 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:47.635347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2ae778e7-d3db-478a-8bce-382527eab676 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:47.636938 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1099/2212] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:54:47 2022] DELETE /identity/v3/projects/af59d2d5839543e6aa2f0eedc5db3f54 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:47.644773 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab5439-3332-497f-805c-fb2147c92a75 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:47.645184 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab5439-3332-497f-805c-fb2147c92a75 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:47.645873 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab5439-3332-497f-805c-fb2147c92a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:47.670193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d8ab5439-3332-497f-805c-fb2147c92a75 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:47.672462 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d8ab5439-3332-497f-805c-fb2147c92a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:47.731659 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1114/2213] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:54:47.741334 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a8193-3861-4d60-98fd-c0aebcc4f569 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:47.741619 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a8193-3861-4d60-98fd-c0aebcc4f569 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:47.745841 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a8193-3861-4d60-98fd-c0aebcc4f569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:47.764085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-414a8193-3861-4d60-98fd-c0aebcc4f569 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:47.765487 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-414a8193-3861-4d60-98fd-c0aebcc4f569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:47.774816 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1100/2214] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:54:47.779192 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab5439-3332-497f-805c-fb2147c92a75 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:47.789573 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab5439-3332-497f-805c-fb2147c92a75 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:47.794314 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6de77-5a84-40a3-933f-0374f8c33d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:47.794933 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6de77-5a84-40a3-933f-0374f8c33d03 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:47.795222 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6de77-5a84-40a3-933f-0374f8c33d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:47.795541 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6de77-5a84-40a3-933f-0374f8c33d03 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:47.797055 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6de77-5a84-40a3-933f-0374f8c33d03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:54:47.797371 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6de77-5a84-40a3-933f-0374f8c33d03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:47.803128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6de77-5a84-40a3-933f-0374f8c33d03 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:47.813135 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1115/2215] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:54:47.816999 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a8193-3861-4d60-98fd-c0aebcc4f569 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:47.826697 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a8193-3861-4d60-98fd-c0aebcc4f569 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:47.832096 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129240b-97d5-4c1b-9fb4-8d3c9d5fb4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:47.832730 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129240b-97d5-4c1b-9fb4-8d3c9d5fb4a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:47.833214 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129240b-97d5-4c1b-9fb4-8d3c9d5fb4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:47.833645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129240b-97d5-4c1b-9fb4-8d3c9d5fb4a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:47.834616 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2129240b-97d5-4c1b-9fb4-8d3c9d5fb4a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:47.845602 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2129240b-97d5-4c1b-9fb4-8d3c9d5fb4a6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:48.019321 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1101/2216] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:54:47 2022] POST /identity/v3/projects => generated 369 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:54:48.027635 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6de77-5a84-40a3-933f-0374f8c33d03 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:48.038391 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6de77-5a84-40a3-933f-0374f8c33d03 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:48.119698 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00795ec-5a9b-4ac1-876d-bd8980ac47db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:48.121279 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00795ec-5a9b-4ac1-876d-bd8980ac47db admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:48.121279 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00795ec-5a9b-4ac1-876d-bd8980ac47db admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:48.121279 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00795ec-5a9b-4ac1-876d-bd8980ac47db admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:48.121779 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00795ec-5a9b-4ac1-876d-bd8980ac47db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:48.125504 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00795ec-5a9b-4ac1-876d-bd8980ac47db admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:48.244198 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1116/2217] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:54:48 2022] POST /identity/v3/users => generated 285 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:54:48.248647 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129240b-97d5-4c1b-9fb4-8d3c9d5fb4a6 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:48.260768 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129240b-97d5-4c1b-9fb4-8d3c9d5fb4a6 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:48.266320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e890f0-75dc-41ad-b1ca-86dde44222b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:48.266938 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e890f0-75dc-41ad-b1ca-86dde44222b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:48.267248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e890f0-75dc-41ad-b1ca-86dde44222b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:48.267551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e890f0-75dc-41ad-b1ca-86dde44222b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:48.268239 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e890f0-75dc-41ad-b1ca-86dde44222b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:48.269903 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e890f0-75dc-41ad-b1ca-86dde44222b9 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:48.280468 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1102/2218] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:54:48.284685 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00795ec-5a9b-4ac1-876d-bd8980ac47db admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:48.299839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00795ec-5a9b-4ac1-876d-bd8980ac47db admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:48.307349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835f2a9-4093-4963-8be5-cc839573fa60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:48.307349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835f2a9-4093-4963-8be5-cc839573fa60 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:48.307349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835f2a9-4093-4963-8be5-cc839573fa60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:48.307349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835f2a9-4093-4963-8be5-cc839573fa60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:48.307727 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835f2a9-4093-4963-8be5-cc839573fa60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:48.308047 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835f2a9-4093-4963-8be5-cc839573fa60 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:48.319318 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1117/2219] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:54:48.324133 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e890f0-75dc-41ad-b1ca-86dde44222b9 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:48.339527 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e890f0-75dc-41ad-b1ca-86dde44222b9 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:48.344157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb8859-7f57-47d7-b1fd-9dbeeaf19b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:48.344892 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb8859-7f57-47d7-b1fd-9dbeeaf19b67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:48.345207 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb8859-7f57-47d7-b1fd-9dbeeaf19b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:48.345521 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb8859-7f57-47d7-b1fd-9dbeeaf19b67 admin admin] PATH_INFO: `/v3/projects/237818e7af49431f91fe72129865b82b/users/1774477286e0429398516ce9a14b96f2/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:48.362844 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb8859-7f57-47d7-b1fd-9dbeeaf19b67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=237818e7af49431f91fe72129865b82b, user_id=1774477286e0429398516ce9a14b96f2, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:48.365798 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb8859-7f57-47d7-b1fd-9dbeeaf19b67 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:48.457626 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1103/2220] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:54:48 2022] PUT /identity/v3/projects/237818e7af49431f91fe72129865b82b/users/1774477286e0429398516ce9a14b96f2/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:48.468313 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb36a02-6b77-4d44-add4-ea967100b8c3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:48.469391 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb36a02-6b77-4d44-add4-ea967100b8c3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:48.469740 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb36a02-6b77-4d44-add4-ea967100b8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:48.533618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-adb36a02-6b77-4d44-add4-ea967100b8c3 None None] MFA Rules not processed for user `1774477286e0429398516ce9a14b96f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:48.535103 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-adb36a02-6b77-4d44-add4-ea967100b8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:48.628876 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1118/2221] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:48 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:54:48.633622 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb8859-7f57-47d7-b1fd-9dbeeaf19b67 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:48.702932 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:48.756582 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb8859-7f57-47d7-b1fd-9dbeeaf19b67 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:48.763463 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845785bd-7944-4713-846e-46c5efb45767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:48.765998 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845785bd-7944-4713-846e-46c5efb45767 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:48.766786 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845785bd-7944-4713-846e-46c5efb45767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:48.767324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb36a02-6b77-4d44-add4-ea967100b8c3 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:48.767892 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845785bd-7944-4713-846e-46c5efb45767 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:48.768787 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-adb36a02-6b77-4d44-add4-ea967100b8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:48.769055 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845785bd-7944-4713-846e-46c5efb45767 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:48.771695 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845785bd-7944-4713-846e-46c5efb45767 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:48.831470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb36a02-6b77-4d44-add4-ea967100b8c3 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:48.836240 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c46159-c232-47b4-9afc-64a722f6420a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:48.836536 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c46159-c232-47b4-9afc-64a722f6420a service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:48.836900 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c46159-c232-47b4-9afc-64a722f6420a service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:48.837216 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c46159-c232-47b4-9afc-64a722f6420a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:48.839940 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c8c46159-c232-47b4-9afc-64a722f6420a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:48.882635 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1104/2222] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:54:48 2022] POST /identity/v3/projects => generated 367 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:54:48.884124 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845785bd-7944-4713-846e-46c5efb45767 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:48.912611 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c46159-c232-47b4-9afc-64a722f6420a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:48.913906 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c46159-c232-47b4-9afc-64a722f6420a service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:48.924725 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1119/2223] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:54:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:54:48.990997 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845785bd-7944-4713-846e-46c5efb45767 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:48.998231 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804f0d1-bfb0-4b6a-84b2-bf4e86634946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:48.999052 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804f0d1-bfb0-4b6a-84b2-bf4e86634946 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:48.999648 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804f0d1-bfb0-4b6a-84b2-bf4e86634946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:49.000226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804f0d1-bfb0-4b6a-84b2-bf4e86634946 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:49.001118 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804f0d1-bfb0-4b6a-84b2-bf4e86634946 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:49.004384 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804f0d1-bfb0-4b6a-84b2-bf4e86634946 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:49.125863 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:49.132991 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1105/2224] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:54:48 2022] POST /identity/v3/users => generated 290 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:54:49.137130 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c46159-c232-47b4-9afc-64a722f6420a service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:49.155643 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c46159-c232-47b4-9afc-64a722f6420a service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:49.161506 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66463b14-f76d-4e5a-b783-873979700c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:49.167096 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66463b14-f76d-4e5a-b783-873979700c40 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:49.167096 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66463b14-f76d-4e5a-b783-873979700c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:49.167096 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66463b14-f76d-4e5a-b783-873979700c40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:49.167999 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66463b14-f76d-4e5a-b783-873979700c40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:49.169637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66463b14-f76d-4e5a-b783-873979700c40 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:49.181421 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1120/2225] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:54:49.185573 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804f0d1-bfb0-4b6a-84b2-bf4e86634946 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:49.195829 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804f0d1-bfb0-4b6a-84b2-bf4e86634946 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:49.200690 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2933c8b-bffc-4c84-b30a-25ef2aa07610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:49.201384 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2933c8b-bffc-4c84-b30a-25ef2aa07610 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:49.201665 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2933c8b-bffc-4c84-b30a-25ef2aa07610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:49.201942 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2933c8b-bffc-4c84-b30a-25ef2aa07610 admin admin] PATH_INFO: `/v3/projects/a1b6e60ecf954c4da6c0fce903c2924d/users/d68a23c1198046c49188a5c768754d57/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:49.217698 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2933c8b-bffc-4c84-b30a-25ef2aa07610 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b6e60ecf954c4da6c0fce903c2924d, user_id=d68a23c1198046c49188a5c768754d57, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:49.221749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2933c8b-bffc-4c84-b30a-25ef2aa07610 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:49.318770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1106/2226] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:54:49 2022] PUT /identity/v3/projects/a1b6e60ecf954c4da6c0fce903c2924d/users/d68a23c1198046c49188a5c768754d57/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:49.323032 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66463b14-f76d-4e5a-b783-873979700c40 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:49.332926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66463b14-f76d-4e5a-b783-873979700c40 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:49.389976 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16326cb7-6914-42cd-bcd1-d5f45576e95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:49.391123 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16326cb7-6914-42cd-bcd1-d5f45576e95e admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:49.391430 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16326cb7-6914-42cd-bcd1-d5f45576e95e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:49.391710 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16326cb7-6914-42cd-bcd1-d5f45576e95e admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:49.392442 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16326cb7-6914-42cd-bcd1-d5f45576e95e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:49.394081 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16326cb7-6914-42cd-bcd1-d5f45576e95e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:49.404436 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1121/2227] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:54:49.409121 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2933c8b-bffc-4c84-b30a-25ef2aa07610 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:49.419339 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2933c8b-bffc-4c84-b30a-25ef2aa07610 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:49.424165 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a654bea-55b0-42be-88a8-d87ac12e3e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:49.424827 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a654bea-55b0-42be-88a8-d87ac12e3e3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:49.425115 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a654bea-55b0-42be-88a8-d87ac12e3e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:49.425393 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a654bea-55b0-42be-88a8-d87ac12e3e3e admin admin] PATH_INFO: `/v3/projects/a1b6e60ecf954c4da6c0fce903c2924d/users/d68a23c1198046c49188a5c768754d57/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:49.427346 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a654bea-55b0-42be-88a8-d87ac12e3e3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b6e60ecf954c4da6c0fce903c2924d, user_id=d68a23c1198046c49188a5c768754d57, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:49.430270 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a654bea-55b0-42be-88a8-d87ac12e3e3e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:49.441031 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1107/2228] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:54:49 2022] PUT /identity/v3/projects/a1b6e60ecf954c4da6c0fce903c2924d/users/d68a23c1198046c49188a5c768754d57/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:49.456454 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545b14c-1256-424f-8bd5-dfcfc0163bcc None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:49.456756 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545b14c-1256-424f-8bd5-dfcfc0163bcc None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:49.457025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545b14c-1256-424f-8bd5-dfcfc0163bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:49.488502 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c545b14c-1256-424f-8bd5-dfcfc0163bcc None None] MFA Rules not processed for user `d68a23c1198046c49188a5c768754d57`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:49.490015 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c545b14c-1256-424f-8bd5-dfcfc0163bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:49.561889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1122/2229] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:49 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:54:49.574668 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a654bea-55b0-42be-88a8-d87ac12e3e3e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:49.579627 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3a654bea-55b0-42be-88a8-d87ac12e3e3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:49.642590 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a654bea-55b0-42be-88a8-d87ac12e3e3e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:49.646831 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69839aab-fb34-4bf4-9711-15889b058dcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:49.647662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69839aab-fb34-4bf4-9711-15889b058dcb service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:49.648112 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69839aab-fb34-4bf4-9711-15889b058dcb service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:49.648507 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69839aab-fb34-4bf4-9711-15889b058dcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:49.663800 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69839aab-fb34-4bf4-9711-15889b058dcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:49.665349 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69839aab-fb34-4bf4-9711-15889b058dcb service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:49.678451 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1108/2230] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:54:49 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:54:49.808251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545b14c-1256-424f-8bd5-dfcfc0163bcc None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:49.820219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545b14c-1256-424f-8bd5-dfcfc0163bcc None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:49.824137 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f0230-18ca-4165-9ec8-5a1f82924274 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:49.824762 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f0230-18ca-4165-9ec8-5a1f82924274 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:49.825043 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f0230-18ca-4165-9ec8-5a1f82924274 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:49.825383 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f0230-18ca-4165-9ec8-5a1f82924274 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:49.891562 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f0230-18ca-4165-9ec8-5a1f82924274 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:49.893024 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f0230-18ca-4165-9ec8-5a1f82924274 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:49.902274 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1123/2231] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:54:49 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:54:51.073182 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:53.867362 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69839aab-fb34-4bf4-9711-15889b058dcb service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:53.869092 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-69839aab-fb34-4bf4-9711-15889b058dcb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:53.879505 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:53.936418 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69839aab-fb34-4bf4-9711-15889b058dcb service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:53.942278 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51093bcc-b9f3-4e82-895e-8a87b6e60a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:53.943011 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51093bcc-b9f3-4e82-895e-8a87b6e60a0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:53.943371 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51093bcc-b9f3-4e82-895e-8a87b6e60a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:53.943746 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51093bcc-b9f3-4e82-895e-8a87b6e60a0e admin admin] PATH_INFO: `/v3/users/e7d4cc581aa044cb8b10758f17f46ad6` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:53.945082 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51093bcc-b9f3-4e82-895e-8a87b6e60a0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7d4cc581aa044cb8b10758f17f46ad6)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:53.947131 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51093bcc-b9f3-4e82-895e-8a87b6e60a0e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:54.166119 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-51093bcc-b9f3-4e82-895e-8a87b6e60a0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7d4cc581aa044cb8b10758f17f46ad6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:54.166793 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-51093bcc-b9f3-4e82-895e-8a87b6e60a0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7d4cc581aa044cb8b10758f17f46ad6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:54.179814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-51093bcc-b9f3-4e82-895e-8a87b6e60a0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7d4cc581aa044cb8b10758f17f46ad6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:54.186795 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-51093bcc-b9f3-4e82-895e-8a87b6e60a0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7d4cc581aa044cb8b10758f17f46ad6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:54.187622 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-51093bcc-b9f3-4e82-895e-8a87b6e60a0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7d4cc581aa044cb8b10758f17f46ad6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:54.294351 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1109/2232] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:54:53 2022] DELETE /identity/v3/users/e7d4cc581aa044cb8b10758f17f46ad6 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:54.299385 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ef048-e69c-425b-aca5-e0aa2a7ac826 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:54.299943 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ef048-e69c-425b-aca5-e0aa2a7ac826 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:54.300101 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ef048-e69c-425b-aca5-e0aa2a7ac826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:54.319613 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-660ef048-e69c-425b-aca5-e0aa2a7ac826 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:54.320859 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-660ef048-e69c-425b-aca5-e0aa2a7ac826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:54.350659 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:54.380620 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1124/2233] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:54:54.390367 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51093bcc-b9f3-4e82-895e-8a87b6e60a0e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:54.391917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-51093bcc-b9f3-4e82-895e-8a87b6e60a0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:54.456551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51093bcc-b9f3-4e82-895e-8a87b6e60a0e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:54.461818 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00229a93-b9e4-4692-b80f-fbc31232cb07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:54.462532 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00229a93-b9e4-4692-b80f-fbc31232cb07 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:54.462850 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00229a93-b9e4-4692-b80f-fbc31232cb07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:54.463100 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00229a93-b9e4-4692-b80f-fbc31232cb07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:54.474771 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00229a93-b9e4-4692-b80f-fbc31232cb07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:54.476324 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00229a93-b9e4-4692-b80f-fbc31232cb07 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:54.488639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1110/2234] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:54:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:54:55.017657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ef048-e69c-425b-aca5-e0aa2a7ac826 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:55.019237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-660ef048-e69c-425b-aca5-e0aa2a7ac826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:55.052909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ef048-e69c-425b-aca5-e0aa2a7ac826 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:55.066766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91e8e7-fbd9-4b0a-ae61-422ccc0e460b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:55.067652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91e8e7-fbd9-4b0a-ae61-422ccc0e460b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:55.079309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91e8e7-fbd9-4b0a-ae61-422ccc0e460b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:55.079640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91e8e7-fbd9-4b0a-ae61-422ccc0e460b admin admin] PATH_INFO: `/v3/projects/b4a8d632792f41d783924c9ff67873c7` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:55.082017 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b91e8e7-fbd9-4b0a-ae61-422ccc0e460b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4a8d632792f41d783924c9ff67873c7)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:55.084003 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b91e8e7-fbd9-4b0a-ae61-422ccc0e460b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:55.256215 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1b91e8e7-fbd9-4b0a-ae61-422ccc0e460b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4a8d632792f41d783924c9ff67873c7', 'request_id': 'req-1b91e8e7-fbd9-4b0a-ae61-422ccc0e460b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:55.261587 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1b91e8e7-fbd9-4b0a-ae61-422ccc0e460b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4a8d632792f41d783924c9ff67873c7', 'request_id': 'req-1b91e8e7-fbd9-4b0a-ae61-422ccc0e460b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:55.392587 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1b91e8e7-fbd9-4b0a-ae61-422ccc0e460b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4a8d632792f41d783924c9ff67873c7', 'request_id': 'req-1b91e8e7-fbd9-4b0a-ae61-422ccc0e460b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:55.404136 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1b91e8e7-fbd9-4b0a-ae61-422ccc0e460b admin admin] The token cache is being invalidate because project b4a8d632792f41d783924c9ff67873c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:54:55.404425 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1b91e8e7-fbd9-4b0a-ae61-422ccc0e460b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:55.405057 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1b91e8e7-fbd9-4b0a-ae61-422ccc0e460b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:55.407192 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1125/2235] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:54:55 2022] DELETE /identity/v3/projects/b4a8d632792f41d783924c9ff67873c7 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:55.411170 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00229a93-b9e4-4692-b80f-fbc31232cb07 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:55.420987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00229a93-b9e4-4692-b80f-fbc31232cb07 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:55.448337 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:55.478894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d2ec4-1139-4f53-9d76-602c71bf793d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:55.479610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d2ec4-1139-4f53-9d76-602c71bf793d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:55.479921 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d2ec4-1139-4f53-9d76-602c71bf793d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:55.480232 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d2ec4-1139-4f53-9d76-602c71bf793d admin admin] PATH_INFO: `/v3/users/348a0cdc31034d518aff9257eda00c9b` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:55.481409 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d2ec4-1139-4f53-9d76-602c71bf793d admin admin] RBAC: Authorizing `identity:delete_user(user_id=348a0cdc31034d518aff9257eda00c9b)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:55.483353 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d2ec4-1139-4f53-9d76-602c71bf793d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:55.755185 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91e8e7-fbd9-4b0a-ae61-422ccc0e460b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:55.756617 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1b91e8e7-fbd9-4b0a-ae61-422ccc0e460b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:55.770738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91e8e7-fbd9-4b0a-ae61-422ccc0e460b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:55.775749 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f2e28-6562-44b3-9110-2fb6e135059a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:55.776397 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f2e28-6562-44b3-9110-2fb6e135059a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:55.776694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f2e28-6562-44b3-9110-2fb6e135059a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:55.776965 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f2e28-6562-44b3-9110-2fb6e135059a admin admin] PATH_INFO: `/v3/users/1774477286e0429398516ce9a14b96f2` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:55.778138 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8f2e28-6562-44b3-9110-2fb6e135059a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1774477286e0429398516ce9a14b96f2)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:55.780108 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8f2e28-6562-44b3-9110-2fb6e135059a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:55.845322 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-de8d2ec4-1139-4f53-9d76-602c71bf793d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '348a0cdc31034d518aff9257eda00c9b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:55.845971 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-de8d2ec4-1139-4f53-9d76-602c71bf793d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '348a0cdc31034d518aff9257eda00c9b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:55.857369 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-de8d2ec4-1139-4f53-9d76-602c71bf793d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '348a0cdc31034d518aff9257eda00c9b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:55.864047 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-de8d2ec4-1139-4f53-9d76-602c71bf793d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '348a0cdc31034d518aff9257eda00c9b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:55.864689 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-de8d2ec4-1139-4f53-9d76-602c71bf793d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '348a0cdc31034d518aff9257eda00c9b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:55.952449 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1111/2236] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:54:55 2022] DELETE /identity/v3/users/348a0cdc31034d518aff9257eda00c9b => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:55.974262 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ee8f2e28-6562-44b3-9110-2fb6e135059a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1774477286e0429398516ce9a14b96f2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:55.974928 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ee8f2e28-6562-44b3-9110-2fb6e135059a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1774477286e0429398516ce9a14b96f2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:55.988749 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ee8f2e28-6562-44b3-9110-2fb6e135059a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1774477286e0429398516ce9a14b96f2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:55.996289 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ee8f2e28-6562-44b3-9110-2fb6e135059a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1774477286e0429398516ce9a14b96f2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:55.997017 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ee8f2e28-6562-44b3-9110-2fb6e135059a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1774477286e0429398516ce9a14b96f2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:56.121114 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1126/2237] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:54:55 2022] DELETE /identity/v3/users/1774477286e0429398516ce9a14b96f2 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:54:56.124574 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3b3ec-64f3-4478-a57a-bec719ab22f4 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:56.124574 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3b3ec-64f3-4478-a57a-bec719ab22f4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:56.124765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3b3ec-64f3-4478-a57a-bec719ab22f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:56.149572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c2d3b3ec-64f3-4478-a57a-bec719ab22f4 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:56.150817 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c2d3b3ec-64f3-4478-a57a-bec719ab22f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:56.214664 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1112/2238] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:54:56.224594 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f2e28-6562-44b3-9110-2fb6e135059a admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:56.226209 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ee8f2e28-6562-44b3-9110-2fb6e135059a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:56.277324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:56.301566 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f2e28-6562-44b3-9110-2fb6e135059a admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:56.305769 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2190909-8e9a-49f2-b996-020f8c132ff6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:56.306831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2190909-8e9a-49f2-b996-020f8c132ff6 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:56.306831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2190909-8e9a-49f2-b996-020f8c132ff6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:56.307667 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2190909-8e9a-49f2-b996-020f8c132ff6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:56.318302 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2190909-8e9a-49f2-b996-020f8c132ff6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:56.320069 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2190909-8e9a-49f2-b996-020f8c132ff6 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:56.335582 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1127/2239] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:54:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:54:56.697213 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3b3ec-64f3-4478-a57a-bec719ab22f4 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:56.698731 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c2d3b3ec-64f3-4478-a57a-bec719ab22f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:56.710814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3b3ec-64f3-4478-a57a-bec719ab22f4 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:56.718101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa04b8b-1e4c-4943-9a84-939af8c53f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:56.718775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa04b8b-1e4c-4943-9a84-939af8c53f0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:56.719063 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa04b8b-1e4c-4943-9a84-939af8c53f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:56.719344 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa04b8b-1e4c-4943-9a84-939af8c53f0a admin admin] PATH_INFO: `/v3/projects/b422aa7594fd4822be9223119ff1ae77` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:56.720576 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa04b8b-1e4c-4943-9a84-939af8c53f0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b422aa7594fd4822be9223119ff1ae77)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:56.723069 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa04b8b-1e4c-4943-9a84-939af8c53f0a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:56.867183 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1fa04b8b-1e4c-4943-9a84-939af8c53f0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b422aa7594fd4822be9223119ff1ae77', 'request_id': 'req-1fa04b8b-1e4c-4943-9a84-939af8c53f0a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:56.872803 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1fa04b8b-1e4c-4943-9a84-939af8c53f0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b422aa7594fd4822be9223119ff1ae77', 'request_id': 'req-1fa04b8b-1e4c-4943-9a84-939af8c53f0a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:56.942162 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2190909-8e9a-49f2-b996-020f8c132ff6 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:56.943635 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c2190909-8e9a-49f2-b996-020f8c132ff6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:56.956535 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2190909-8e9a-49f2-b996-020f8c132ff6 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:57.014861 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1fa04b8b-1e4c-4943-9a84-939af8c53f0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b422aa7594fd4822be9223119ff1ae77', 'request_id': 'req-1fa04b8b-1e4c-4943-9a84-939af8c53f0a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:57.017684 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fc1af-921d-4ea7-a1ce-53a1947e1519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:57.017853 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fc1af-921d-4ea7-a1ce-53a1947e1519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:57.018049 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fc1af-921d-4ea7-a1ce-53a1947e1519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:57.018323 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fc1af-921d-4ea7-a1ce-53a1947e1519 admin admin] PATH_INFO: `/v3/projects/237818e7af49431f91fe72129865b82b` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:57.019599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68fc1af-921d-4ea7-a1ce-53a1947e1519 admin admin] RBAC: Authorizing `identity:delete_project(project_id=237818e7af49431f91fe72129865b82b)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:57.021473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68fc1af-921d-4ea7-a1ce-53a1947e1519 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:57.027191 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1fa04b8b-1e4c-4943-9a84-939af8c53f0a admin admin] The token cache is being invalidate because project b422aa7594fd4822be9223119ff1ae77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:54:57.027517 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1fa04b8b-1e4c-4943-9a84-939af8c53f0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:57.028167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1fa04b8b-1e4c-4943-9a84-939af8c53f0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:57.029741 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1113/2240] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:54:56 2022] DELETE /identity/v3/projects/b422aa7594fd4822be9223119ff1ae77 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:57.034075 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa04b8b-1e4c-4943-9a84-939af8c53f0a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:57.035488 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1fa04b8b-1e4c-4943-9a84-939af8c53f0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:57.054560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa04b8b-1e4c-4943-9a84-939af8c53f0a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:57.059657 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35d86b-0bb6-4339-941c-6ded107afb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:57.060312 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35d86b-0bb6-4339-941c-6ded107afb57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:57.060627 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35d86b-0bb6-4339-941c-6ded107afb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:57.060908 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35d86b-0bb6-4339-941c-6ded107afb57 admin admin] PATH_INFO: `/v3/users/4a2e27fc22d746ac8f8ff348219bc486` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:57.062093 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd35d86b-0bb6-4339-941c-6ded107afb57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a2e27fc22d746ac8f8ff348219bc486)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:57.064069 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd35d86b-0bb6-4339-941c-6ded107afb57 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:57.164319 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f68fc1af-921d-4ea7-a1ce-53a1947e1519 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '237818e7af49431f91fe72129865b82b', 'request_id': 'req-f68fc1af-921d-4ea7-a1ce-53a1947e1519'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:57.170067 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f68fc1af-921d-4ea7-a1ce-53a1947e1519 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '237818e7af49431f91fe72129865b82b', 'request_id': 'req-f68fc1af-921d-4ea7-a1ce-53a1947e1519'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:57.209223 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:57.266872 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f68fc1af-921d-4ea7-a1ce-53a1947e1519 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '237818e7af49431f91fe72129865b82b', 'request_id': 'req-f68fc1af-921d-4ea7-a1ce-53a1947e1519'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:57.278679 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f68fc1af-921d-4ea7-a1ce-53a1947e1519 admin admin] The token cache is being invalidate because project 237818e7af49431f91fe72129865b82b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:54:57.278971 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f68fc1af-921d-4ea7-a1ce-53a1947e1519 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:57.279671 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f68fc1af-921d-4ea7-a1ce-53a1947e1519 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:57.281221 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1128/2241] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:54:56 2022] DELETE /identity/v3/projects/237818e7af49431f91fe72129865b82b => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:54:57.285111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fc1af-921d-4ea7-a1ce-53a1947e1519 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:57.286422 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f68fc1af-921d-4ea7-a1ce-53a1947e1519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:57.311334 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-dd35d86b-0bb6-4339-941c-6ded107afb57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a2e27fc22d746ac8f8ff348219bc486'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:57.312028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-dd35d86b-0bb6-4339-941c-6ded107afb57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a2e27fc22d746ac8f8ff348219bc486'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:57.324575 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-dd35d86b-0bb6-4339-941c-6ded107afb57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a2e27fc22d746ac8f8ff348219bc486'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:57.331513 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-dd35d86b-0bb6-4339-941c-6ded107afb57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a2e27fc22d746ac8f8ff348219bc486'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:57.332215 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-dd35d86b-0bb6-4339-941c-6ded107afb57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a2e27fc22d746ac8f8ff348219bc486'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:57.365393 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fc1af-921d-4ea7-a1ce-53a1947e1519 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:57.370967 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4309464-8623-49e8-ad0a-cd0b56816f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:57.371730 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4309464-8623-49e8-ad0a-cd0b56816f30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:57.372013 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4309464-8623-49e8-ad0a-cd0b56816f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:57.372285 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4309464-8623-49e8-ad0a-cd0b56816f30 admin admin] PATH_INFO: `/v3/users/d68a23c1198046c49188a5c768754d57` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:57.373485 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4309464-8623-49e8-ad0a-cd0b56816f30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d68a23c1198046c49188a5c768754d57)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:57.376197 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4309464-8623-49e8-ad0a-cd0b56816f30 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:57.443946 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1114/2242] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:54:57 2022] DELETE /identity/v3/users/4a2e27fc22d746ac8f8ff348219bc486 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:54:57.833692 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c4309464-8623-49e8-ad0a-cd0b56816f30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd68a23c1198046c49188a5c768754d57'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:57.834568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c4309464-8623-49e8-ad0a-cd0b56816f30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd68a23c1198046c49188a5c768754d57'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:57.848390 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c4309464-8623-49e8-ad0a-cd0b56816f30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd68a23c1198046c49188a5c768754d57'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:57.856312 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c4309464-8623-49e8-ad0a-cd0b56816f30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd68a23c1198046c49188a5c768754d57'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:57.857515 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c4309464-8623-49e8-ad0a-cd0b56816f30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd68a23c1198046c49188a5c768754d57'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:57.945991 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1129/2243] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:54:57 2022] DELETE /identity/v3/users/d68a23c1198046c49188a5c768754d57 => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:54:58.074541 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35d86b-0bb6-4339-941c-6ded107afb57 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:58.076054 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-dd35d86b-0bb6-4339-941c-6ded107afb57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:58.154204 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35d86b-0bb6-4339-941c-6ded107afb57 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:58.213269 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5d104-8428-42d9-a10c-b377c06d57a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:58.213914 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5d104-8428-42d9-a10c-b377c06d57a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:58.214199 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5d104-8428-42d9-a10c-b377c06d57a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:58.214479 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5d104-8428-42d9-a10c-b377c06d57a8 admin admin] PATH_INFO: `/v3/projects/57651e1b425849dda5c655426b9462dd` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:58.215682 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a5d104-8428-42d9-a10c-b377c06d57a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57651e1b425849dda5c655426b9462dd)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:58.217625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a5d104-8428-42d9-a10c-b377c06d57a8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:58.384973 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f3a5d104-8428-42d9-a10c-b377c06d57a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57651e1b425849dda5c655426b9462dd', 'request_id': 'req-f3a5d104-8428-42d9-a10c-b377c06d57a8'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:58.391675 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f3a5d104-8428-42d9-a10c-b377c06d57a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57651e1b425849dda5c655426b9462dd', 'request_id': 'req-f3a5d104-8428-42d9-a10c-b377c06d57a8'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:58.466391 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f3a5d104-8428-42d9-a10c-b377c06d57a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57651e1b425849dda5c655426b9462dd', 'request_id': 'req-f3a5d104-8428-42d9-a10c-b377c06d57a8'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:58.478145 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f3a5d104-8428-42d9-a10c-b377c06d57a8 admin admin] The token cache is being invalidate because project 57651e1b425849dda5c655426b9462dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:54:58.478436 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f3a5d104-8428-42d9-a10c-b377c06d57a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:58.480110 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f3a5d104-8428-42d9-a10c-b377c06d57a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:58.480807 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1115/2244] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:54:58 2022] DELETE /identity/v3/projects/57651e1b425849dda5c655426b9462dd => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:58.486447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8bb22-70e6-425d-948e-59f15cb80dd7 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:58.486741 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8bb22-70e6-425d-948e-59f15cb80dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:58.487026 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8bb22-70e6-425d-948e-59f15cb80dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:58.502877 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:58.509748 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d6b8bb22-70e6-425d-948e-59f15cb80dd7 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:58.511013 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d6b8bb22-70e6-425d-948e-59f15cb80dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:58.566791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5d104-8428-42d9-a10c-b377c06d57a8 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:58.568511 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f3a5d104-8428-42d9-a10c-b377c06d57a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:58.583147 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1130/2245] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:54:58.589475 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc10d5-cd29-4e5c-a241-7300b2b4c623 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:58.589763 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc10d5-cd29-4e5c-a241-7300b2b4c623 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:58.590064 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc10d5-cd29-4e5c-a241-7300b2b4c623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:58.611528 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f4bc10d5-cd29-4e5c-a241-7300b2b4c623 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:58.612764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f4bc10d5-cd29-4e5c-a241-7300b2b4c623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:58.621038 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1131/2246] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:54:58.626227 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc10d5-cd29-4e5c-a241-7300b2b4c623 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:58.637273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc10d5-cd29-4e5c-a241-7300b2b4c623 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:58.642045 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a1ae66-addc-4eb9-a079-0ba6f6d731ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:58.642663 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a1ae66-addc-4eb9-a079-0ba6f6d731ea admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:58.642966 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a1ae66-addc-4eb9-a079-0ba6f6d731ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:58.643249 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a1ae66-addc-4eb9-a079-0ba6f6d731ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:58.643949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a1ae66-addc-4eb9-a079-0ba6f6d731ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:54:58.644251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a1ae66-addc-4eb9-a079-0ba6f6d731ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:58.645831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a1ae66-addc-4eb9-a079-0ba6f6d731ea admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:58.655992 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5d104-8428-42d9-a10c-b377c06d57a8 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:58.659439 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1132/2247] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:54:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:54:58.665957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a1ae66-addc-4eb9-a079-0ba6f6d731ea admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:58.666274 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a69e9-2a17-4833-be44-054fb7a6b2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:58.666274 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a69e9-2a17-4833-be44-054fb7a6b2d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:58.666274 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a69e9-2a17-4833-be44-054fb7a6b2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:58.666274 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a69e9-2a17-4833-be44-054fb7a6b2d2 admin admin] PATH_INFO: `/v3/projects/a1b6e60ecf954c4da6c0fce903c2924d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:58.668864 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a69e9-2a17-4833-be44-054fb7a6b2d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1b6e60ecf954c4da6c0fce903c2924d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:58.671321 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a69e9-2a17-4833-be44-054fb7a6b2d2 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:58.673821 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a1ae66-addc-4eb9-a079-0ba6f6d731ea admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:58.679290 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b68a7-7ae0-4c71-b10c-7c150535df13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:58.680303 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b68a7-7ae0-4c71-b10c-7c150535df13 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:58.680303 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b68a7-7ae0-4c71-b10c-7c150535df13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:58.680482 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b68a7-7ae0-4c71-b10c-7c150535df13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:58.681154 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349b68a7-7ae0-4c71-b10c-7c150535df13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:58.683214 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349b68a7-7ae0-4c71-b10c-7c150535df13 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:58.802444 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b99a69e9-2a17-4833-be44-054fb7a6b2d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1b6e60ecf954c4da6c0fce903c2924d', 'request_id': 'req-b99a69e9-2a17-4833-be44-054fb7a6b2d2'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:58.810217 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b99a69e9-2a17-4833-be44-054fb7a6b2d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1b6e60ecf954c4da6c0fce903c2924d', 'request_id': 'req-b99a69e9-2a17-4833-be44-054fb7a6b2d2'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:58.817610 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1133/2248] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:54:58 2022] POST /identity/v3/projects => generated 383 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:54:58.821856 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b68a7-7ae0-4c71-b10c-7c150535df13 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:58.884053 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b68a7-7ae0-4c71-b10c-7c150535df13 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:58.888844 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13afb89f-b3ca-47df-bb8a-bc7fae56b912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:58.889461 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13afb89f-b3ca-47df-bb8a-bc7fae56b912 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:58.890007 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13afb89f-b3ca-47df-bb8a-bc7fae56b912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:58.890282 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13afb89f-b3ca-47df-bb8a-bc7fae56b912 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:58.890984 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13afb89f-b3ca-47df-bb8a-bc7fae56b912 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:58.893029 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13afb89f-b3ca-47df-bb8a-bc7fae56b912 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:58.940423 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b99a69e9-2a17-4833-be44-054fb7a6b2d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1b6e60ecf954c4da6c0fce903c2924d', 'request_id': 'req-b99a69e9-2a17-4833-be44-054fb7a6b2d2'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:58.952641 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b99a69e9-2a17-4833-be44-054fb7a6b2d2 admin admin] The token cache is being invalidate because project a1b6e60ecf954c4da6c0fce903c2924d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:54:58.952935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b99a69e9-2a17-4833-be44-054fb7a6b2d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:58.953615 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b99a69e9-2a17-4833-be44-054fb7a6b2d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:54:58.955255 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1116/2249] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:54:58 2022] DELETE /identity/v3/projects/a1b6e60ecf954c4da6c0fce903c2924d => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:54:58.961357 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436e700-4bbf-41a0-b952-a8cddc79cd73 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:58.961647 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436e700-4bbf-41a0-b952-a8cddc79cd73 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:58.961935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436e700-4bbf-41a0-b952-a8cddc79cd73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:58.976567 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1134/2250] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:54:58 2022] POST /identity/v3/users => generated 292 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:54:58.980231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13afb89f-b3ca-47df-bb8a-bc7fae56b912 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:58.981602 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-2436e700-4bbf-41a0-b952-a8cddc79cd73 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:58.981850 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-13afb89f-b3ca-47df-bb8a-bc7fae56b912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:58.982796 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2436e700-4bbf-41a0-b952-a8cddc79cd73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:58.991620 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1117/2251] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:54:58.994278 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13afb89f-b3ca-47df-bb8a-bc7fae56b912 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:58.998944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8c9c0-7726-4df4-8c1b-1e52b006232e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:58.999235 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8c9c0-7726-4df4-8c1b-1e52b006232e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:58.999489 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049530d2-c0bf-4d0c-8f73-cb9fc69609e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:58.999636 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8c9c0-7726-4df4-8c1b-1e52b006232e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:59.000750 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049530d2-c0bf-4d0c-8f73-cb9fc69609e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:59.001035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049530d2-c0bf-4d0c-8f73-cb9fc69609e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:59.001487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049530d2-c0bf-4d0c-8f73-cb9fc69609e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:59.002340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049530d2-c0bf-4d0c-8f73-cb9fc69609e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:59.003986 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049530d2-c0bf-4d0c-8f73-cb9fc69609e5 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:59.014389 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1135/2252] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:54:59.020390 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049530d2-c0bf-4d0c-8f73-cb9fc69609e5 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:59.020520 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-74d8c9c0-7726-4df4-8c1b-1e52b006232e None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:59.021636 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-74d8c9c0-7726-4df4-8c1b-1e52b006232e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:59.029606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1118/2253] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:54:59.032801 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049530d2-c0bf-4d0c-8f73-cb9fc69609e5 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:59.034041 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d8c9c0-7726-4df4-8c1b-1e52b006232e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:59.042076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb409a1-cd67-4c1f-852a-0e6033d781f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:59.042076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb409a1-cd67-4c1f-852a-0e6033d781f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:59.042076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb409a1-cd67-4c1f-852a-0e6033d781f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:59.042076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb409a1-cd67-4c1f-852a-0e6033d781f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:59.043943 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb409a1-cd67-4c1f-852a-0e6033d781f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:59.043943 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb409a1-cd67-4c1f-852a-0e6033d781f9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:59.044106 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d8c9c0-7726-4df4-8c1b-1e52b006232e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:59.048931 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf4094-c8a9-4bdb-94bb-cdce7c0892b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:59.049656 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf4094-c8a9-4bdb-94bb-cdce7c0892b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:59.050076 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf4094-c8a9-4bdb-94bb-cdce7c0892b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:59.050355 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf4094-c8a9-4bdb-94bb-cdce7c0892b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:59.051052 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf4094-c8a9-4bdb-94bb-cdce7c0892b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:54:59.051357 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf4094-c8a9-4bdb-94bb-cdce7c0892b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:59.053008 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf4094-c8a9-4bdb-94bb-cdce7c0892b5 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:59.054309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1136/2254] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:54:59.058219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb409a1-cd67-4c1f-852a-0e6033d781f9 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:59.065788 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1119/2255] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:54:59.068621 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb409a1-cd67-4c1f-852a-0e6033d781f9 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:59.070034 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf4094-c8a9-4bdb-94bb-cdce7c0892b5 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:59.073541 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc798f-95ec-4205-b142-42ffd1674114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:59.074206 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc798f-95ec-4205-b142-42ffd1674114 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:59.074492 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc798f-95ec-4205-b142-42ffd1674114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:59.074770 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc798f-95ec-4205-b142-42ffd1674114 admin admin] PATH_INFO: `/v3/projects/32d3753b5f524ab78fd777c9c9c7b9ed/users/973ce853a0894326894cba4eb44813a0/roles/e033961a9363456a8510c34414dd839e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:59.080977 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf4094-c8a9-4bdb-94bb-cdce7c0892b5 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:59.086177 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62279aeb-5df9-450e-90cd-a8414e2c8fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:59.086997 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62279aeb-5df9-450e-90cd-a8414e2c8fab admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:59.087240 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62279aeb-5df9-450e-90cd-a8414e2c8fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:59.087686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62279aeb-5df9-450e-90cd-a8414e2c8fab admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:59.088350 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62279aeb-5df9-450e-90cd-a8414e2c8fab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:59.090274 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62279aeb-5df9-450e-90cd-a8414e2c8fab admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:59.091360 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc798f-95ec-4205-b142-42ffd1674114 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32d3753b5f524ab78fd777c9c9c7b9ed, user_id=973ce853a0894326894cba4eb44813a0, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:59.095027 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc798f-95ec-4205-b142-42ffd1674114 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:59.107811 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:59.203479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1137/2256] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:54:59 2022] PUT /identity/v3/projects/32d3753b5f524ab78fd777c9c9c7b9ed/users/973ce853a0894326894cba4eb44813a0/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:54:59.215302 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a08e9-0896-4103-a14c-4fccece5c6bd None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:59.215620 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a08e9-0896-4103-a14c-4fccece5c6bd None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:59.215899 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a08e9-0896-4103-a14c-4fccece5c6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:59.237164 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1120/2257] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:54:59 2022] POST /identity/v3/projects => generated 385 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:54:59.238075 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62279aeb-5df9-450e-90cd-a8414e2c8fab admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:59.244678 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a69a08e9-0896-4103-a14c-4fccece5c6bd None None] MFA Rules not processed for user `973ce853a0894326894cba4eb44813a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:59.245875 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a69a08e9-0896-4103-a14c-4fccece5c6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:59.300212 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62279aeb-5df9-450e-90cd-a8414e2c8fab admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:59.305633 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5332e2-deeb-48f6-a0be-e47d570828d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:59.306277 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5332e2-deeb-48f6-a0be-e47d570828d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:59.306544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5332e2-deeb-48f6-a0be-e47d570828d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:59.306822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5332e2-deeb-48f6-a0be-e47d570828d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:59.307515 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5332e2-deeb-48f6-a0be-e47d570828d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:59.313825 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5332e2-deeb-48f6-a0be-e47d570828d3 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:59.314082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1138/2258] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:54:59.314791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a08e9-0896-4103-a14c-4fccece5c6bd None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:59.325487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a08e9-0896-4103-a14c-4fccece5c6bd None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:59.329867 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e187188-0233-4c11-8a84-0795b480d184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:59.330594 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e187188-0233-4c11-8a84-0795b480d184 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:59.330848 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e187188-0233-4c11-8a84-0795b480d184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:59.331129 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e187188-0233-4c11-8a84-0795b480d184 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:59.331820 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e187188-0233-4c11-8a84-0795b480d184 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:59.333756 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e187188-0233-4c11-8a84-0795b480d184 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:59.438915 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1121/2259] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:54:59 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:54:59.442609 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1139/2260] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:54:59 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:54:59.443701 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e187188-0233-4c11-8a84-0795b480d184 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:59.443824 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5332e2-deeb-48f6-a0be-e47d570828d3 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:59.456847 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5332e2-deeb-48f6-a0be-e47d570828d3 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:59.513688 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e187188-0233-4c11-8a84-0795b480d184 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:59.518461 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4b909-11fa-4493-a4e4-3331b21aee4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:59.519087 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4b909-11fa-4493-a4e4-3331b21aee4c admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:59.519378 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4b909-11fa-4493-a4e4-3331b21aee4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:59.519856 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4b909-11fa-4493-a4e4-3331b21aee4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:59.519971 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c40fd-3827-430e-b7db-ffc1b3fd5231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:59.520599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c40fd-3827-430e-b7db-ffc1b3fd5231 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:59.520888 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c40fd-3827-430e-b7db-ffc1b3fd5231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:59.521164 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c40fd-3827-430e-b7db-ffc1b3fd5231 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:59.521832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5c40fd-3827-430e-b7db-ffc1b3fd5231 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:59.524987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5c40fd-3827-430e-b7db-ffc1b3fd5231 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:59.528224 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4b909-11fa-4493-a4e4-3331b21aee4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:59.530127 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4b909-11fa-4493-a4e4-3331b21aee4c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:59.546839 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1122/2261] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:59 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:54:59.547974 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4b909-11fa-4493-a4e4-3331b21aee4c admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:59.557960 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4b909-11fa-4493-a4e4-3331b21aee4c admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:59.563076 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dfebe8-47c0-4338-a407-ff6751919269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:59.563733 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dfebe8-47c0-4338-a407-ff6751919269 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:59.564022 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dfebe8-47c0-4338-a407-ff6751919269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:59.564300 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dfebe8-47c0-4338-a407-ff6751919269 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:59.564994 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dfebe8-47c0-4338-a407-ff6751919269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:59.566648 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dfebe8-47c0-4338-a407-ff6751919269 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:59.580802 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1123/2262] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:54:59.585028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dfebe8-47c0-4338-a407-ff6751919269 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:59.594848 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dfebe8-47c0-4338-a407-ff6751919269 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:59.599622 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a511a-6e69-45e6-b3f0-d3c4c897a2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:59.600291 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a511a-6e69-45e6-b3f0-d3c4c897a2c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:59.600574 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a511a-6e69-45e6-b3f0-d3c4c897a2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:59.600852 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a511a-6e69-45e6-b3f0-d3c4c897a2c1 admin admin] PATH_INFO: `/v3/projects/ddad58ee91f04e7ba94bbc930a6ad3fd/users/6f5b3dfdbb1f41bf820f69083773faf3/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:59.617040 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a511a-6e69-45e6-b3f0-d3c4c897a2c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddad58ee91f04e7ba94bbc930a6ad3fd, user_id=6f5b3dfdbb1f41bf820f69083773faf3, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:59.619736 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a511a-6e69-45e6-b3f0-d3c4c897a2c1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:59.661059 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1140/2263] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:54:59 2022] POST /identity/v3/users => generated 299 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:54:59.664981 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c40fd-3827-430e-b7db-ffc1b3fd5231 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:59.675257 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c40fd-3827-430e-b7db-ffc1b3fd5231 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:59.680490 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b58db4c-b9e0-4b9a-b5cb-3a4ddf48b8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:59.681118 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58db4c-b9e0-4b9a-b5cb-3a4ddf48b8f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:59.681403 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58db4c-b9e0-4b9a-b5cb-3a4ddf48b8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:59.681836 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58db4c-b9e0-4b9a-b5cb-3a4ddf48b8f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:59.682530 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b58db4c-b9e0-4b9a-b5cb-3a4ddf48b8f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:59.684163 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b58db4c-b9e0-4b9a-b5cb-3a4ddf48b8f2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:59.694757 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1141/2264] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:54:59.699546 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b58db4c-b9e0-4b9a-b5cb-3a4ddf48b8f2 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:59.709139 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b58db4c-b9e0-4b9a-b5cb-3a4ddf48b8f2 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:59.713766 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b873a4f-d293-427a-9be7-c4061cf8ece5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:59.714406 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b873a4f-d293-427a-9be7-c4061cf8ece5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:59.714699 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b873a4f-d293-427a-9be7-c4061cf8ece5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:59.714980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b873a4f-d293-427a-9be7-c4061cf8ece5 admin admin] PATH_INFO: `/v3/projects/095ce9a40fa04425998d94e3d5d91624/users/a88acee527f140fbaccc8fae9c221902/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:59.729792 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b873a4f-d293-427a-9be7-c4061cf8ece5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=095ce9a40fa04425998d94e3d5d91624, user_id=a88acee527f140fbaccc8fae9c221902, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:59.732477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b873a4f-d293-427a-9be7-c4061cf8ece5 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:59.758216 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1124/2265] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:54:59 2022] PUT /identity/v3/projects/ddad58ee91f04e7ba94bbc930a6ad3fd/users/6f5b3dfdbb1f41bf820f69083773faf3/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:54:59.768675 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4963b9-2deb-4bda-b714-6830684b3006 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:59.768969 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4963b9-2deb-4bda-b714-6830684b3006 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:59.769252 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4963b9-2deb-4bda-b714-6830684b3006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:59.796190 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6e4963b9-2deb-4bda-b714-6830684b3006 None None] MFA Rules not processed for user `6f5b3dfdbb1f41bf820f69083773faf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:59.798866 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6e4963b9-2deb-4bda-b714-6830684b3006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:59.855662 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1142/2266] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:54:59 2022] PUT /identity/v3/projects/095ce9a40fa04425998d94e3d5d91624/users/a88acee527f140fbaccc8fae9c221902/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:54:59.860202 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b873a4f-d293-427a-9be7-c4061cf8ece5 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:59.871054 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1125/2267] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:54:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:54:59.872927 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4963b9-2deb-4bda-b714-6830684b3006 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:59.949247 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b873a4f-d293-427a-9be7-c4061cf8ece5 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:59.959517 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de86923f-ea25-4af3-9ae5-3e72a65c4c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:59.960346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de86923f-ea25-4af3-9ae5-3e72a65c4c87 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:59.960346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de86923f-ea25-4af3-9ae5-3e72a65c4c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:59.960618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de86923f-ea25-4af3-9ae5-3e72a65c4c87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:59.961322 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de86923f-ea25-4af3-9ae5-3e72a65c4c87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:59.963834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de86923f-ea25-4af3-9ae5-3e72a65c4c87 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:59.968064 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:59.974222 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4963b9-2deb-4bda-b714-6830684b3006 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:59.979229 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e2b88-a624-437d-8aba-576a749578c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:59.980101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e2b88-a624-437d-8aba-576a749578c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:59.980238 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e2b88-a624-437d-8aba-576a749578c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:59.980447 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e2b88-a624-437d-8aba-576a749578c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:59.981114 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2e2b88-a624-437d-8aba-576a749578c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:59.983038 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2e2b88-a624-437d-8aba-576a749578c1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:59.989662 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1143/2268] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:54:59 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:54:59.991998 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de86923f-ea25-4af3-9ae5-3e72a65c4c87 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:00.007477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de86923f-ea25-4af3-9ae5-3e72a65c4c87 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:00.013399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700dad6-fc73-4c6b-b734-3e5ce8e26a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:00.014072 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700dad6-fc73-4c6b-b734-3e5ce8e26a9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:00.014356 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700dad6-fc73-4c6b-b734-3e5ce8e26a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:00.014885 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700dad6-fc73-4c6b-b734-3e5ce8e26a9a admin admin] PATH_INFO: `/v3/projects/095ce9a40fa04425998d94e3d5d91624/users/a88acee527f140fbaccc8fae9c221902/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:00.016954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9700dad6-fc73-4c6b-b734-3e5ce8e26a9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=095ce9a40fa04425998d94e3d5d91624, user_id=a88acee527f140fbaccc8fae9c221902, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:00.019692 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9700dad6-fc73-4c6b-b734-3e5ce8e26a9a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:00.030937 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1144/2269] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:54:59 2022] PUT /identity/v3/projects/095ce9a40fa04425998d94e3d5d91624/users/a88acee527f140fbaccc8fae9c221902/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:00.041719 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a0004-0cad-421a-ba7e-fe26463ed324 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:00.042029 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a0004-0cad-421a-ba7e-fe26463ed324 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:00.042290 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a0004-0cad-421a-ba7e-fe26463ed324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:00.074586 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1126/2270] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:54:59 2022] POST /identity/v3/projects => generated 387 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:55:00.079199 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-5d8a0004-0cad-421a-ba7e-fe26463ed324 None None] MFA Rules not processed for user `a88acee527f140fbaccc8fae9c221902`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:00.079199 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5d8a0004-0cad-421a-ba7e-fe26463ed324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:00.080273 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e2b88-a624-437d-8aba-576a749578c1 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:00.142427 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e2b88-a624-437d-8aba-576a749578c1 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:00.146241 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1145/2271] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:55:00.147939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcc1da1-d381-4948-9200-89d39ea59712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:00.148639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc1da1-d381-4948-9200-89d39ea59712 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:00.148991 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc1da1-d381-4948-9200-89d39ea59712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:00.149335 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc1da1-d381-4948-9200-89d39ea59712 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:00.150108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcc1da1-d381-4948-9200-89d39ea59712 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:00.152094 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcc1da1-d381-4948-9200-89d39ea59712 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:00.157343 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8a0004-0cad-421a-ba7e-fe26463ed324 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:00.158684 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5d8a0004-0cad-421a-ba7e-fe26463ed324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:00.214890 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8a0004-0cad-421a-ba7e-fe26463ed324 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:00.218563 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0d0d2-6b9e-4ac4-bd9c-53f3eb56c8d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:00.219703 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0d0d2-6b9e-4ac4-bd9c-53f3eb56c8d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:00.219989 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0d0d2-6b9e-4ac4-bd9c-53f3eb56c8d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:00.220263 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0d0d2-6b9e-4ac4-bd9c-53f3eb56c8d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:00.249084 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1127/2272] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:55:00 2022] POST /identity/v3/users => generated 300 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:55:00.252977 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcc1da1-d381-4948-9200-89d39ea59712 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:00.263097 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcc1da1-d381-4948-9200-89d39ea59712 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:00.269227 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037820eb-8c61-4bef-aeb8-0aec452e21db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:00.269822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037820eb-8c61-4bef-aeb8-0aec452e21db admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:00.270109 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037820eb-8c61-4bef-aeb8-0aec452e21db admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:00.282108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037820eb-8c61-4bef-aeb8-0aec452e21db admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:00.282108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037820eb-8c61-4bef-aeb8-0aec452e21db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:00.282108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037820eb-8c61-4bef-aeb8-0aec452e21db admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:00.282359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca0d0d2-6b9e-4ac4-bd9c-53f3eb56c8d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:00.282359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca0d0d2-6b9e-4ac4-bd9c-53f3eb56c8d2 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:00.286515 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1128/2273] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:55:00.287624 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037820eb-8c61-4bef-aeb8-0aec452e21db admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:00.288949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1146/2274] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:55:00.297756 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037820eb-8c61-4bef-aeb8-0aec452e21db admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:00.302616 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac81326-676b-41a9-98e6-fea87adb4ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:00.303277 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac81326-676b-41a9-98e6-fea87adb4ecb admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:00.303599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac81326-676b-41a9-98e6-fea87adb4ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:00.303877 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac81326-676b-41a9-98e6-fea87adb4ecb admin admin] PATH_INFO: `/v3/projects/78e47ec1fb2b472daa9c10c6688fd0a9/users/26cd9b852f814b96a32e9fd93eb064d5/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:00.318608 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac81326-676b-41a9-98e6-fea87adb4ecb admin admin] RBAC: Authorizing `identity:create_grant(project_id=78e47ec1fb2b472daa9c10c6688fd0a9, user_id=26cd9b852f814b96a32e9fd93eb064d5, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:00.321333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac81326-676b-41a9-98e6-fea87adb4ecb admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:00.410235 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1129/2275] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:55:00 2022] PUT /identity/v3/projects/78e47ec1fb2b472daa9c10c6688fd0a9/users/26cd9b852f814b96a32e9fd93eb064d5/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:55:00.416050 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0d0d2-6b9e-4ac4-bd9c-53f3eb56c8d2 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:00.420914 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:00.427715 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0d0d2-6b9e-4ac4-bd9c-53f3eb56c8d2 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:00.485704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5491615-f7be-480e-be21-bdae6c8a695e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:00.486244 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5491615-f7be-480e-be21-bdae6c8a695e admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:00.486547 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5491615-f7be-480e-be21-bdae6c8a695e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:00.486820 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5491615-f7be-480e-be21-bdae6c8a695e admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:00.487556 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5491615-f7be-480e-be21-bdae6c8a695e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:00.489253 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5491615-f7be-480e-be21-bdae6c8a695e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:00.499231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1147/2276] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:55:00.503466 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac81326-676b-41a9-98e6-fea87adb4ecb admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:00.513640 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac81326-676b-41a9-98e6-fea87adb4ecb admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:00.519065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de570b-4540-4db0-93d8-9332464a24cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:00.519772 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de570b-4540-4db0-93d8-9332464a24cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:00.520061 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de570b-4540-4db0-93d8-9332464a24cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:00.520339 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de570b-4540-4db0-93d8-9332464a24cc admin admin] PATH_INFO: `/v3/projects/78e47ec1fb2b472daa9c10c6688fd0a9/users/26cd9b852f814b96a32e9fd93eb064d5/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:00.522311 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de570b-4540-4db0-93d8-9332464a24cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=78e47ec1fb2b472daa9c10c6688fd0a9, user_id=26cd9b852f814b96a32e9fd93eb064d5, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:00.524985 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de570b-4540-4db0-93d8-9332464a24cc admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:00.535510 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1130/2277] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:55:00 2022] PUT /identity/v3/projects/78e47ec1fb2b472daa9c10c6688fd0a9/users/26cd9b852f814b96a32e9fd93eb064d5/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:00.546471 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b3dab-30d4-49a0-bba9-6737316f6b95 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:00.558305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b3dab-30d4-49a0-bba9-6737316f6b95 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:00.558305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b3dab-30d4-49a0-bba9-6737316f6b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:00.574569 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ea0b3dab-30d4-49a0-bba9-6737316f6b95 None None] MFA Rules not processed for user `26cd9b852f814b96a32e9fd93eb064d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:00.575869 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ea0b3dab-30d4-49a0-bba9-6737316f6b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:00.646924 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1148/2278] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:00 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:00.659126 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de570b-4540-4db0-93d8-9332464a24cc admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:00.714721 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de570b-4540-4db0-93d8-9332464a24cc admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:00.718364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dbe493-52d9-4df9-8e37-7300fe792d70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:00.719012 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbe493-52d9-4df9-8e37-7300fe792d70 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:00.719297 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbe493-52d9-4df9-8e37-7300fe792d70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:00.719612 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbe493-52d9-4df9-8e37-7300fe792d70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:00.729190 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dbe493-52d9-4df9-8e37-7300fe792d70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:00.733086 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dbe493-52d9-4df9-8e37-7300fe792d70 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:00.744869 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1131/2279] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:00 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:55:00.774347 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b3dab-30d4-49a0-bba9-6737316f6b95 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:00.785753 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:00.841787 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b3dab-30d4-49a0-bba9-6737316f6b95 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:00.846777 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff43328-4127-4b0f-95e8-0aed63c5392a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:00.847464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff43328-4127-4b0f-95e8-0aed63c5392a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:00.847754 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff43328-4127-4b0f-95e8-0aed63c5392a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:00.848034 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff43328-4127-4b0f-95e8-0aed63c5392a admin admin] PATH_INFO: `/v3/users/6f5b3dfdbb1f41bf820f69083773faf3` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:00.849235 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff43328-4127-4b0f-95e8-0aed63c5392a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f5b3dfdbb1f41bf820f69083773faf3)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:00.851175 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff43328-4127-4b0f-95e8-0aed63c5392a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:01.105459 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-dff43328-4127-4b0f-95e8-0aed63c5392a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f5b3dfdbb1f41bf820f69083773faf3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:01.106824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-dff43328-4127-4b0f-95e8-0aed63c5392a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f5b3dfdbb1f41bf820f69083773faf3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:01.121216 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-dff43328-4127-4b0f-95e8-0aed63c5392a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f5b3dfdbb1f41bf820f69083773faf3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:01.130242 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-dff43328-4127-4b0f-95e8-0aed63c5392a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f5b3dfdbb1f41bf820f69083773faf3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:01.131182 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-dff43328-4127-4b0f-95e8-0aed63c5392a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f5b3dfdbb1f41bf820f69083773faf3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:01.255014 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1149/2280] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:55:00 2022] DELETE /identity/v3/users/6f5b3dfdbb1f41bf820f69083773faf3 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:01.262221 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca8f87-fe18-4f7c-ae3a-4d69298c387a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:01.262696 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca8f87-fe18-4f7c-ae3a-4d69298c387a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:01.262778 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca8f87-fe18-4f7c-ae3a-4d69298c387a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:01.289119 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c5ca8f87-fe18-4f7c-ae3a-4d69298c387a None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:01.290343 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c5ca8f87-fe18-4f7c-ae3a-4d69298c387a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:01.364834 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:01.366278 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1132/2281] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:01.374268 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff43328-4127-4b0f-95e8-0aed63c5392a admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:01.374566 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-dff43328-4127-4b0f-95e8-0aed63c5392a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:01.441401 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff43328-4127-4b0f-95e8-0aed63c5392a admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:01.445676 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267288bb-2327-479e-8aac-e83f09ca4393 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:01.446312 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267288bb-2327-479e-8aac-e83f09ca4393 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:01.446596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267288bb-2327-479e-8aac-e83f09ca4393 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:01.446876 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267288bb-2327-479e-8aac-e83f09ca4393 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:01.458610 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267288bb-2327-479e-8aac-e83f09ca4393 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:01.463778 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267288bb-2327-479e-8aac-e83f09ca4393 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:01.472098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1150/2282] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:55:01.638127 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:01.954285 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca8f87-fe18-4f7c-ae3a-4d69298c387a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:01.956173 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c5ca8f87-fe18-4f7c-ae3a-4d69298c387a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:01.983776 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca8f87-fe18-4f7c-ae3a-4d69298c387a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:01.983776 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8248c89-f22c-47cb-b6f6-220e6b88f51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:01.983776 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8248c89-f22c-47cb-b6f6-220e6b88f51d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:01.983776 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8248c89-f22c-47cb-b6f6-220e6b88f51d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:01.984265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8248c89-f22c-47cb-b6f6-220e6b88f51d admin admin] PATH_INFO: `/v3/projects/ddad58ee91f04e7ba94bbc930a6ad3fd` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:01.984265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8248c89-f22c-47cb-b6f6-220e6b88f51d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddad58ee91f04e7ba94bbc930a6ad3fd)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:01.984265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8248c89-f22c-47cb-b6f6-220e6b88f51d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:02.106679 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c8248c89-f22c-47cb-b6f6-220e6b88f51d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddad58ee91f04e7ba94bbc930a6ad3fd', 'request_id': 'req-c8248c89-f22c-47cb-b6f6-220e6b88f51d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:02.112522 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c8248c89-f22c-47cb-b6f6-220e6b88f51d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddad58ee91f04e7ba94bbc930a6ad3fd', 'request_id': 'req-c8248c89-f22c-47cb-b6f6-220e6b88f51d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:02.134851 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:02.214430 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c8248c89-f22c-47cb-b6f6-220e6b88f51d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddad58ee91f04e7ba94bbc930a6ad3fd', 'request_id': 'req-c8248c89-f22c-47cb-b6f6-220e6b88f51d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:02.227193 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c8248c89-f22c-47cb-b6f6-220e6b88f51d admin admin] The token cache is being invalidate because project ddad58ee91f04e7ba94bbc930a6ad3fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:55:02.227517 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c8248c89-f22c-47cb-b6f6-220e6b88f51d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:02.228170 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c8248c89-f22c-47cb-b6f6-220e6b88f51d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:02.229708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1133/2283] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:55:01 2022] DELETE /identity/v3/projects/ddad58ee91f04e7ba94bbc930a6ad3fd => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:02.233783 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267288bb-2327-479e-8aac-e83f09ca4393 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:02.245484 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267288bb-2327-479e-8aac-e83f09ca4393 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:02.309802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae46d4-9766-4293-bdb3-dbc63047ee05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:02.310460 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae46d4-9766-4293-bdb3-dbc63047ee05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:02.310777 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae46d4-9766-4293-bdb3-dbc63047ee05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:02.311089 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae46d4-9766-4293-bdb3-dbc63047ee05 admin admin] PATH_INFO: `/v3/users/26cd9b852f814b96a32e9fd93eb064d5` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:02.312337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eae46d4-9766-4293-bdb3-dbc63047ee05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26cd9b852f814b96a32e9fd93eb064d5)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:02.314331 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eae46d4-9766-4293-bdb3-dbc63047ee05 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:02.362606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:03.005773 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4eae46d4-9766-4293-bdb3-dbc63047ee05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26cd9b852f814b96a32e9fd93eb064d5'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:03.006458 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4eae46d4-9766-4293-bdb3-dbc63047ee05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26cd9b852f814b96a32e9fd93eb064d5'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:03.019691 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4eae46d4-9766-4293-bdb3-dbc63047ee05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26cd9b852f814b96a32e9fd93eb064d5'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:03.027195 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4eae46d4-9766-4293-bdb3-dbc63047ee05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26cd9b852f814b96a32e9fd93eb064d5'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:03.027946 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4eae46d4-9766-4293-bdb3-dbc63047ee05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26cd9b852f814b96a32e9fd93eb064d5'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:03.135474 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1151/2284] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:55:02 2022] DELETE /identity/v3/users/26cd9b852f814b96a32e9fd93eb064d5 => generated 0 bytes in 894 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:03.189578 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:03.347453 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:03.596454 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8248c89-f22c-47cb-b6f6-220e6b88f51d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:03.598046 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c8248c89-f22c-47cb-b6f6-220e6b88f51d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:03.615563 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8248c89-f22c-47cb-b6f6-220e6b88f51d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:03.677604 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8816aa-22da-422f-a929-3268c81d3d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:03.678256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8816aa-22da-422f-a929-3268c81d3d24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:03.678552 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8816aa-22da-422f-a929-3268c81d3d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:03.679347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8816aa-22da-422f-a929-3268c81d3d24 admin admin] PATH_INFO: `/v3/projects/78e47ec1fb2b472daa9c10c6688fd0a9` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:03.680760 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8816aa-22da-422f-a929-3268c81d3d24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78e47ec1fb2b472daa9c10c6688fd0a9)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:03.682829 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8816aa-22da-422f-a929-3268c81d3d24 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:03.783722 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ef8816aa-22da-422f-a929-3268c81d3d24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78e47ec1fb2b472daa9c10c6688fd0a9', 'request_id': 'req-ef8816aa-22da-422f-a929-3268c81d3d24'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:03.789070 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ef8816aa-22da-422f-a929-3268c81d3d24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78e47ec1fb2b472daa9c10c6688fd0a9', 'request_id': 'req-ef8816aa-22da-422f-a929-3268c81d3d24'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:03.900665 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ef8816aa-22da-422f-a929-3268c81d3d24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78e47ec1fb2b472daa9c10c6688fd0a9', 'request_id': 'req-ef8816aa-22da-422f-a929-3268c81d3d24'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:03.913071 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ef8816aa-22da-422f-a929-3268c81d3d24 admin admin] The token cache is being invalidate because project 78e47ec1fb2b472daa9c10c6688fd0a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:55:03.915882 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ef8816aa-22da-422f-a929-3268c81d3d24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:03.915882 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ef8816aa-22da-422f-a929-3268c81d3d24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:03.915882 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1134/2285] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:55:03 2022] DELETE /identity/v3/projects/78e47ec1fb2b472daa9c10c6688fd0a9 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:03.920679 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf8215-da8d-4fb9-8ff0-c6b7d28fa698 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:03.921172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf8215-da8d-4fb9-8ff0-c6b7d28fa698 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:03.921295 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf8215-da8d-4fb9-8ff0-c6b7d28fa698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:03.944840 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-fecf8215-da8d-4fb9-8ff0-c6b7d28fa698 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:03.944840 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fecf8215-da8d-4fb9-8ff0-c6b7d28fa698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:04.004324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1152/2286] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:55:04.011916 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b41f6d-596e-404f-9c75-6a7f34304161 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:04.012473 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b41f6d-596e-404f-9c75-6a7f34304161 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:04.012908 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b41f6d-596e-404f-9c75-6a7f34304161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:04.042709 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-22b41f6d-596e-404f-9c75-6a7f34304161 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:04.042709 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-22b41f6d-596e-404f-9c75-6a7f34304161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:04.044760 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1135/2287] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:04.049751 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf8215-da8d-4fb9-8ff0-c6b7d28fa698 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:04.059816 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf8215-da8d-4fb9-8ff0-c6b7d28fa698 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:04.065204 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa46eb-ad28-42a9-8ca9-86699ed0d016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:04.066008 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa46eb-ad28-42a9-8ca9-86699ed0d016 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:04.066444 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa46eb-ad28-42a9-8ca9-86699ed0d016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:04.066761 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa46eb-ad28-42a9-8ca9-86699ed0d016 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:04.067497 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa46eb-ad28-42a9-8ca9-86699ed0d016 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:55:04.069933 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa46eb-ad28-42a9-8ca9-86699ed0d016 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:04.069933 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa46eb-ad28-42a9-8ca9-86699ed0d016 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:04.082671 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1153/2288] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:55:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:55:04.087065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b41f6d-596e-404f-9c75-6a7f34304161 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:04.096116 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b41f6d-596e-404f-9c75-6a7f34304161 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:04.100849 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e9f481-5ad9-4c83-a54c-e968ff314959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:04.101533 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9f481-5ad9-4c83-a54c-e968ff314959 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:04.101816 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9f481-5ad9-4c83-a54c-e968ff314959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:04.102085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9f481-5ad9-4c83-a54c-e968ff314959 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:04.102754 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e9f481-5ad9-4c83-a54c-e968ff314959 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:04.104720 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e9f481-5ad9-4c83-a54c-e968ff314959 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:04.218984 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1136/2289] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:55:04 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:55:04.220417 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa46eb-ad28-42a9-8ca9-86699ed0d016 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:04.231437 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa46eb-ad28-42a9-8ca9-86699ed0d016 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:04.252400 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:04.322168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48190ec7-ec81-4a83-8f67-d1c151178205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:04.326754 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48190ec7-ec81-4a83-8f67-d1c151178205 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:04.326754 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48190ec7-ec81-4a83-8f67-d1c151178205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:04.326754 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48190ec7-ec81-4a83-8f67-d1c151178205 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:04.326754 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48190ec7-ec81-4a83-8f67-d1c151178205 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:04.326754 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48190ec7-ec81-4a83-8f67-d1c151178205 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:04.448874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1154/2290] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:55:04 2022] POST /identity/v3/users => generated 293 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:55:04.450409 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e9f481-5ad9-4c83-a54c-e968ff314959 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:04.463458 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e9f481-5ad9-4c83-a54c-e968ff314959 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:04.477933 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc8128-89bc-41b5-b42a-d6444dc4d3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:04.477933 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc8128-89bc-41b5-b42a-d6444dc4d3e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:04.477933 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc8128-89bc-41b5-b42a-d6444dc4d3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:04.477933 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc8128-89bc-41b5-b42a-d6444dc4d3e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:04.478303 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc8128-89bc-41b5-b42a-d6444dc4d3e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:04.478549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc8128-89bc-41b5-b42a-d6444dc4d3e9 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:04.495639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1137/2291] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:55:04.499556 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48190ec7-ec81-4a83-8f67-d1c151178205 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:04.510147 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48190ec7-ec81-4a83-8f67-d1c151178205 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:04.515470 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0baf4-404a-448d-b9c2-e1ab8f82dd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:04.516227 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0baf4-404a-448d-b9c2-e1ab8f82dd01 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:04.516584 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0baf4-404a-448d-b9c2-e1ab8f82dd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:04.516937 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0baf4-404a-448d-b9c2-e1ab8f82dd01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:04.517700 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba0baf4-404a-448d-b9c2-e1ab8f82dd01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:04.519359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba0baf4-404a-448d-b9c2-e1ab8f82dd01 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:04.530299 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1155/2292] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:55:04.534996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc8128-89bc-41b5-b42a-d6444dc4d3e9 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:04.545660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc8128-89bc-41b5-b42a-d6444dc4d3e9 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:04.551102 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d1878-74bc-4c6c-b59b-ba8835f0ac05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:04.551915 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d1878-74bc-4c6c-b59b-ba8835f0ac05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:04.552295 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d1878-74bc-4c6c-b59b-ba8835f0ac05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:04.552641 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d1878-74bc-4c6c-b59b-ba8835f0ac05 admin admin] PATH_INFO: `/v3/projects/7881897199d9488c882683856d8f7cf2/users/165ee02e256b4eadb556b4c4c8e6633f/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:04.578288 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d1878-74bc-4c6c-b59b-ba8835f0ac05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7881897199d9488c882683856d8f7cf2, user_id=165ee02e256b4eadb556b4c4c8e6633f, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:04.581470 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d1878-74bc-4c6c-b59b-ba8835f0ac05 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:04.714967 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1138/2293] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:55:04 2022] PUT /identity/v3/projects/7881897199d9488c882683856d8f7cf2/users/165ee02e256b4eadb556b4c4c8e6633f/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:55:04.725519 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dec7a8-32c5-4c16-87b1-fe95308a45aa None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:04.725880 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dec7a8-32c5-4c16-87b1-fe95308a45aa None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:04.726075 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dec7a8-32c5-4c16-87b1-fe95308a45aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:04.758854 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f0dec7a8-32c5-4c16-87b1-fe95308a45aa None None] MFA Rules not processed for user `165ee02e256b4eadb556b4c4c8e6633f`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:04.760118 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f0dec7a8-32c5-4c16-87b1-fe95308a45aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:04.861593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1156/2294] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:04.866024 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d1878-74bc-4c6c-b59b-ba8835f0ac05 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:04.916526 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:04.981273 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d1878-74bc-4c6c-b59b-ba8835f0ac05 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:04.986358 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c200c-d050-4526-8667-182b346135e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:04.986998 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c200c-d050-4526-8667-182b346135e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:04.987283 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c200c-d050-4526-8667-182b346135e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:04.987602 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c200c-d050-4526-8667-182b346135e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:04.988308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87c200c-d050-4526-8667-182b346135e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:04.990331 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87c200c-d050-4526-8667-182b346135e1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:05.066515 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1139/2295] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:55:04 2022] POST /identity/v3/projects => generated 387 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:55:05.066884 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dec7a8-32c5-4c16-87b1-fe95308a45aa None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:05.077345 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dec7a8-32c5-4c16-87b1-fe95308a45aa None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:05.134260 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8832ce-2f29-4b64-9b03-2495e618c8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:05.134890 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8832ce-2f29-4b64-9b03-2495e618c8e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:05.135258 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8832ce-2f29-4b64-9b03-2495e618c8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:05.135562 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8832ce-2f29-4b64-9b03-2495e618c8e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:05.140483 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8832ce-2f29-4b64-9b03-2495e618c8e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:05.140483 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8832ce-2f29-4b64-9b03-2495e618c8e1 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:05.278370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1157/2296] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:55:05 2022] POST /identity/v3/users => generated 300 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:55:05.284066 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c200c-d050-4526-8667-182b346135e1 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:05.293016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c200c-d050-4526-8667-182b346135e1 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:05.298232 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a8f0f-9bed-4595-abda-1a5a8b4a1f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:05.298849 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a8f0f-9bed-4595-abda-1a5a8b4a1f89 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:05.299135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a8f0f-9bed-4595-abda-1a5a8b4a1f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:05.299447 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a8f0f-9bed-4595-abda-1a5a8b4a1f89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:05.300136 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a8f0f-9bed-4595-abda-1a5a8b4a1f89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:05.301747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a8f0f-9bed-4595-abda-1a5a8b4a1f89 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:05.313959 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1140/2297] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:55:05.321220 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8832ce-2f29-4b64-9b03-2495e618c8e1 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:05.331785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8832ce-2f29-4b64-9b03-2495e618c8e1 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:05.336446 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdff69b-4f13-4b6c-b068-40cd3e6cee87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:05.337109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdff69b-4f13-4b6c-b068-40cd3e6cee87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:05.337394 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdff69b-4f13-4b6c-b068-40cd3e6cee87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:05.337673 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdff69b-4f13-4b6c-b068-40cd3e6cee87 admin admin] PATH_INFO: `/v3/projects/5e1d544cfa96414f9197c8f4172eef68/users/918b55fbff594672a887f158fe2f0083/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:05.353981 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdff69b-4f13-4b6c-b068-40cd3e6cee87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e1d544cfa96414f9197c8f4172eef68, user_id=918b55fbff594672a887f158fe2f0083, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:05.356682 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdff69b-4f13-4b6c-b068-40cd3e6cee87 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:05.360552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:05.478503 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1158/2298] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:55:05 2022] PUT /identity/v3/projects/5e1d544cfa96414f9197c8f4172eef68/users/918b55fbff594672a887f158fe2f0083/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:05.482552 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a8f0f-9bed-4595-abda-1a5a8b4a1f89 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:05.492694 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a8f0f-9bed-4595-abda-1a5a8b4a1f89 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:05.526975 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdff69b-4f13-4b6c-b068-40cd3e6cee87 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:05.528897 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3bdff69b-4f13-4b6c-b068-40cd3e6cee87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:05.552049 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42ff2b-a692-4825-a83b-07bd63e91ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:05.552672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42ff2b-a692-4825-a83b-07bd63e91ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:05.552953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42ff2b-a692-4825-a83b-07bd63e91ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:05.553228 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42ff2b-a692-4825-a83b-07bd63e91ef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:05.553913 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc42ff2b-a692-4825-a83b-07bd63e91ef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:05.555528 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc42ff2b-a692-4825-a83b-07bd63e91ef2 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:05.566234 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1141/2299] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:55:05.570573 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42ff2b-a692-4825-a83b-07bd63e91ef2 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:05.580644 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42ff2b-a692-4825-a83b-07bd63e91ef2 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:05.585329 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beddf3b1-38a9-4554-aa04-8e056997c9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:05.585979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beddf3b1-38a9-4554-aa04-8e056997c9b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:05.586263 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beddf3b1-38a9-4554-aa04-8e056997c9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:05.586536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beddf3b1-38a9-4554-aa04-8e056997c9b5 admin admin] PATH_INFO: `/v3/projects/5e1d544cfa96414f9197c8f4172eef68/users/918b55fbff594672a887f158fe2f0083/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:05.588479 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beddf3b1-38a9-4554-aa04-8e056997c9b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e1d544cfa96414f9197c8f4172eef68, user_id=918b55fbff594672a887f158fe2f0083, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:05.591258 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beddf3b1-38a9-4554-aa04-8e056997c9b5 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:05.601903 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1142/2300] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:55:05 2022] PUT /identity/v3/projects/5e1d544cfa96414f9197c8f4172eef68/users/918b55fbff594672a887f158fe2f0083/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:55:05.606023 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdff69b-4f13-4b6c-b068-40cd3e6cee87 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:05.613592 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b132f3-effc-4887-8920-bd086355d49e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:05.614050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b132f3-effc-4887-8920-bd086355d49e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:05.614425 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b132f3-effc-4887-8920-bd086355d49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:05.645850 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e9b132f3-effc-4887-8920-bd086355d49e None None] MFA Rules not processed for user `918b55fbff594672a887f158fe2f0083`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:05.647070 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e9b132f3-effc-4887-8920-bd086355d49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:05.676049 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c271f-69d5-4c9a-8727-e1e833cf14af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '252c2d2939134416a3e8efb254f03c5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:05.676820 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c271f-69d5-4c9a-8727-e1e833cf14af service placement] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:05.677225 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c271f-69d5-4c9a-8727-e1e833cf14af service placement] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:05.677797 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c271f-69d5-4c9a-8727-e1e833cf14af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:05.679611 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5a6c271f-69d5-4c9a-8727-e1e833cf14af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:05.696188 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c271f-69d5-4c9a-8727-e1e833cf14af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:05.698189 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c271f-69d5-4c9a-8727-e1e833cf14af service placement] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:05.713167 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1159/2301] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:55:05.729263 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1143/2302] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:05 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:55:05.740493 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c271f-69d5-4c9a-8727-e1e833cf14af service placement] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:05.747053 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5a6c271f-69d5-4c9a-8727-e1e833cf14af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:05.843575 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c271f-69d5-4c9a-8727-e1e833cf14af service placement] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:05.849782 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b0bfb-3305-402f-a080-a942f737a0c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:05.849782 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b0bfb-3305-402f-a080-a942f737a0c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:05.849782 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b0bfb-3305-402f-a080-a942f737a0c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:05.849782 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b0bfb-3305-402f-a080-a942f737a0c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:05.858305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080b0bfb-3305-402f-a080-a942f737a0c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:05.860486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080b0bfb-3305-402f-a080-a942f737a0c9 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:05.871074 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1160/2303] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:05 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:55:07.569492 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b132f3-effc-4887-8920-bd086355d49e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:07.575175 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:07.634862 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b132f3-effc-4887-8920-bd086355d49e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:07.639567 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71175220-d377-4990-88f6-5226110eab97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:07.640469 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71175220-d377-4990-88f6-5226110eab97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:07.640749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71175220-d377-4990-88f6-5226110eab97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:07.641112 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71175220-d377-4990-88f6-5226110eab97 admin admin] PATH_INFO: `/v3/users/165ee02e256b4eadb556b4c4c8e6633f` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:07.642312 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71175220-d377-4990-88f6-5226110eab97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=165ee02e256b4eadb556b4c4c8e6633f)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:07.644328 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71175220-d377-4990-88f6-5226110eab97 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:07.909353 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-71175220-d377-4990-88f6-5226110eab97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '165ee02e256b4eadb556b4c4c8e6633f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:07.910215 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-71175220-d377-4990-88f6-5226110eab97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '165ee02e256b4eadb556b4c4c8e6633f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:07.929955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-71175220-d377-4990-88f6-5226110eab97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '165ee02e256b4eadb556b4c4c8e6633f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:07.944567 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-71175220-d377-4990-88f6-5226110eab97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '165ee02e256b4eadb556b4c4c8e6633f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:07.945544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-71175220-d377-4990-88f6-5226110eab97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '165ee02e256b4eadb556b4c4c8e6633f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:08.066999 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ed729-f888-43ed-84cd-763fe28a6945 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:08.067558 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ed729-f888-43ed-84cd-763fe28a6945 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:08.070035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ed729-f888-43ed-84cd-763fe28a6945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:08.070664 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1144/2304] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:55:07 2022] DELETE /identity/v3/users/165ee02e256b4eadb556b4c4c8e6633f => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:08.099664 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-173ed729-f888-43ed-84cd-763fe28a6945 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:08.101002 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-173ed729-f888-43ed-84cd-763fe28a6945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:08.129500 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:08.177118 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1161/2305] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:08.187484 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71175220-d377-4990-88f6-5226110eab97 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:08.189477 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-71175220-d377-4990-88f6-5226110eab97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:08.252638 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71175220-d377-4990-88f6-5226110eab97 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:08.256777 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd0df7-13b4-4f7a-9709-4e178f596cb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:08.258610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd0df7-13b4-4f7a-9709-4e178f596cb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:08.259094 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd0df7-13b4-4f7a-9709-4e178f596cb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:08.259570 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd0df7-13b4-4f7a-9709-4e178f596cb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:08.270735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd0df7-13b4-4f7a-9709-4e178f596cb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:08.273640 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd0df7-13b4-4f7a-9709-4e178f596cb8 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:08.291645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1145/2306] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:55:08.632328 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72015933-c12c-44a2-9d16-e7a4c6ca01c4 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:08.632619 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72015933-c12c-44a2-9d16-e7a4c6ca01c4 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:08.634266 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72015933-c12c-44a2-9d16-e7a4c6ca01c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:08.656308 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-72015933-c12c-44a2-9d16-e7a4c6ca01c4 None None] MFA Rules not processed for user `5179c3c4eaaa41f3beb79ba7f1993478`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:08.657583 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-72015933-c12c-44a2-9d16-e7a4c6ca01c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:08.716088 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1162/2307] 10.222.0.97 () {58 vars in 1009 bytes} [Tue Jul 26 14:55:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:08.761000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd0df7-13b4-4f7a-9709-4e178f596cb8 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:08.762591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-41cd0df7-13b4-4f7a-9709-4e178f596cb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:08.833880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd0df7-13b4-4f7a-9709-4e178f596cb8 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:08.839340 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e39d98-5263-4c6c-afc4-69088460fba5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85122c4052fd4ba19a2dae185a707caf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:08.839870 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e39d98-5263-4c6c-afc4-69088460fba5 service swift] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:08.840398 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e39d98-5263-4c6c-afc4-69088460fba5 service swift] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:08.840398 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e39d98-5263-4c6c-afc4-69088460fba5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:08.849030 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e39d98-5263-4c6c-afc4-69088460fba5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:08.850760 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e39d98-5263-4c6c-afc4-69088460fba5 service swift] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:08.860762 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1146/2308] 10.222.0.97 () {60 vars in 1384 bytes} [Tue Jul 26 14:55:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 14:55:08.866572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:09.154968 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72015933-c12c-44a2-9d16-e7a4c6ca01c4 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:09.158345 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-72015933-c12c-44a2-9d16-e7a4c6ca01c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:09.174786 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72015933-c12c-44a2-9d16-e7a4c6ca01c4 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:09.184337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74f394-ca4e-4516-bb91-b82eeab5d6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:09.184337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74f394-ca4e-4516-bb91-b82eeab5d6ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:09.184337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74f394-ca4e-4516-bb91-b82eeab5d6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:09.184337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74f394-ca4e-4516-bb91-b82eeab5d6ff admin admin] PATH_INFO: `/v3/projects/7881897199d9488c882683856d8f7cf2` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:09.185098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb74f394-ca4e-4516-bb91-b82eeab5d6ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=7881897199d9488c882683856d8f7cf2)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:09.186019 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb74f394-ca4e-4516-bb91-b82eeab5d6ff admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:09.359809 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fb74f394-ca4e-4516-bb91-b82eeab5d6ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7881897199d9488c882683856d8f7cf2', 'request_id': 'req-fb74f394-ca4e-4516-bb91-b82eeab5d6ff'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:09.366803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fb74f394-ca4e-4516-bb91-b82eeab5d6ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7881897199d9488c882683856d8f7cf2', 'request_id': 'req-fb74f394-ca4e-4516-bb91-b82eeab5d6ff'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:09.566504 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fb74f394-ca4e-4516-bb91-b82eeab5d6ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7881897199d9488c882683856d8f7cf2', 'request_id': 'req-fb74f394-ca4e-4516-bb91-b82eeab5d6ff'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:09.586224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fb74f394-ca4e-4516-bb91-b82eeab5d6ff admin admin] The token cache is being invalidate because project 7881897199d9488c882683856d8f7cf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:55:09.586523 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fb74f394-ca4e-4516-bb91-b82eeab5d6ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:09.587248 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fb74f394-ca4e-4516-bb91-b82eeab5d6ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:09.590050 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1163/2309] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:55:09 2022] DELETE /identity/v3/projects/7881897199d9488c882683856d8f7cf2 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:09.594719 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e39d98-5263-4c6c-afc4-69088460fba5 service swift] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:09.608522 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e39d98-5263-4c6c-afc4-69088460fba5 service swift] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:09.648999 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74f394-ca4e-4516-bb91-b82eeab5d6ff admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:09.651506 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fb74f394-ca4e-4516-bb91-b82eeab5d6ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:09.711722 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81cb78-bb3b-4180-97f5-e6208852516f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:09.712576 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81cb78-bb3b-4180-97f5-e6208852516f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:09.713040 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81cb78-bb3b-4180-97f5-e6208852516f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:09.714108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81cb78-bb3b-4180-97f5-e6208852516f admin admin] PATH_INFO: `/v3/users/918b55fbff594672a887f158fe2f0083` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:09.716213 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81cb78-bb3b-4180-97f5-e6208852516f admin admin] RBAC: Authorizing `identity:delete_user(user_id=918b55fbff594672a887f158fe2f0083)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:09.719739 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81cb78-bb3b-4180-97f5-e6208852516f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:09.756402 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74f394-ca4e-4516-bb91-b82eeab5d6ff admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:09.760320 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49839ab8-b475-454f-a12a-0e6714dcd64c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:09.761003 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49839ab8-b475-454f-a12a-0e6714dcd64c service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:09.761288 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49839ab8-b475-454f-a12a-0e6714dcd64c service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:09.761564 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49839ab8-b475-454f-a12a-0e6714dcd64c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:09.763368 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-49839ab8-b475-454f-a12a-0e6714dcd64c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:09.774528 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:09.833167 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49839ab8-b475-454f-a12a-0e6714dcd64c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:09.834638 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49839ab8-b475-454f-a12a-0e6714dcd64c service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:09.847886 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1164/2310] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:09 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:55:09.917552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49839ab8-b475-454f-a12a-0e6714dcd64c service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:09.919050 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-49839ab8-b475-454f-a12a-0e6714dcd64c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:09.977104 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49839ab8-b475-454f-a12a-0e6714dcd64c service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:09.981271 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef066b96-835b-4b93-a5be-2d02476bdb43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:09.981890 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef066b96-835b-4b93-a5be-2d02476bdb43 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:09.982171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef066b96-835b-4b93-a5be-2d02476bdb43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:09.982442 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef066b96-835b-4b93-a5be-2d02476bdb43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:09.984259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ef066b96-835b-4b93-a5be-2d02476bdb43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:10.051370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef066b96-835b-4b93-a5be-2d02476bdb43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:10.052906 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef066b96-835b-4b93-a5be-2d02476bdb43 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:10.062731 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1165/2311] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:09 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:55:10.139179 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-dc81cb78-bb3b-4180-97f5-e6208852516f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '918b55fbff594672a887f158fe2f0083'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:10.139867 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-dc81cb78-bb3b-4180-97f5-e6208852516f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '918b55fbff594672a887f158fe2f0083'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:10.152409 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-dc81cb78-bb3b-4180-97f5-e6208852516f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '918b55fbff594672a887f158fe2f0083'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:10.159155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-dc81cb78-bb3b-4180-97f5-e6208852516f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '918b55fbff594672a887f158fe2f0083'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:10.159832 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-dc81cb78-bb3b-4180-97f5-e6208852516f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '918b55fbff594672a887f158fe2f0083'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:10.267329 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1147/2312] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:55:09 2022] DELETE /identity/v3/users/918b55fbff594672a887f158fe2f0083 => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:10.852443 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef066b96-835b-4b93-a5be-2d02476bdb43 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:10.856980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ef066b96-835b-4b93-a5be-2d02476bdb43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:10.867895 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef066b96-835b-4b93-a5be-2d02476bdb43 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:10.915866 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab0ded-04f5-4a7b-8c9e-dcd9d0a17766 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:10.916160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab0ded-04f5-4a7b-8c9e-dcd9d0a17766 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:10.916441 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab0ded-04f5-4a7b-8c9e-dcd9d0a17766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:10.939359 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9eab0ded-04f5-4a7b-8c9e-dcd9d0a17766 None None] MFA Rules not processed for user `5179c3c4eaaa41f3beb79ba7f1993478`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:10.943028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9eab0ded-04f5-4a7b-8c9e-dcd9d0a17766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:10.947723 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f5a680-17d0-4c6a-be87-32b3a959815a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:10.948622 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5a680-17d0-4c6a-be87-32b3a959815a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:10.948909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5a680-17d0-4c6a-be87-32b3a959815a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:10.949186 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5a680-17d0-4c6a-be87-32b3a959815a admin admin] PATH_INFO: `/v3/projects/5e1d544cfa96414f9197c8f4172eef68` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:10.950362 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f5a680-17d0-4c6a-be87-32b3a959815a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e1d544cfa96414f9197c8f4172eef68)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:10.952457 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f5a680-17d0-4c6a-be87-32b3a959815a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:11.000291 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1148/2313] 10.222.0.97 () {58 vars in 967 bytes} [Tue Jul 26 14:55:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:55:11.013439 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eab0ded-04f5-4a7b-8c9e-dcd9d0a17766 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:11.014954 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9eab0ded-04f5-4a7b-8c9e-dcd9d0a17766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:11.091142 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eab0ded-04f5-4a7b-8c9e-dcd9d0a17766 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:11.098028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a7e10-4059-48df-8405-73d886fece17 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85122c4052fd4ba19a2dae185a707caf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:11.098643 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a7e10-4059-48df-8405-73d886fece17 service swift] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:11.098936 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a7e10-4059-48df-8405-73d886fece17 service swift] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:11.099273 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a7e10-4059-48df-8405-73d886fece17 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:11.110067 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a7e10-4059-48df-8405-73d886fece17 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:11.111711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a7e10-4059-48df-8405-73d886fece17 service swift] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:11.120145 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1149/2314] 10.222.0.97 () {60 vars in 1384 bytes} [Tue Jul 26 14:55:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 14:55:11.190799 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e2f5a680-17d0-4c6a-be87-32b3a959815a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e1d544cfa96414f9197c8f4172eef68', 'request_id': 'req-e2f5a680-17d0-4c6a-be87-32b3a959815a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:11.196441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e2f5a680-17d0-4c6a-be87-32b3a959815a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e1d544cfa96414f9197c8f4172eef68', 'request_id': 'req-e2f5a680-17d0-4c6a-be87-32b3a959815a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:11.307183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e2f5a680-17d0-4c6a-be87-32b3a959815a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e1d544cfa96414f9197c8f4172eef68', 'request_id': 'req-e2f5a680-17d0-4c6a-be87-32b3a959815a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:11.320233 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e2f5a680-17d0-4c6a-be87-32b3a959815a admin admin] The token cache is being invalidate because project 5e1d544cfa96414f9197c8f4172eef68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:55:11.329740 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e2f5a680-17d0-4c6a-be87-32b3a959815a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:11.329740 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e2f5a680-17d0-4c6a-be87-32b3a959815a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:11.329740 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1166/2315] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:55:10 2022] DELETE /identity/v3/projects/5e1d544cfa96414f9197c8f4172eef68 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:11.330030 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67863e1-60de-4e66-8dc5-ba9ccf96a093 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:11.330030 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67863e1-60de-4e66-8dc5-ba9ccf96a093 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:11.330030 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67863e1-60de-4e66-8dc5-ba9ccf96a093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:11.349057 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a67863e1-60de-4e66-8dc5-ba9ccf96a093 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:11.350320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a67863e1-60de-4e66-8dc5-ba9ccf96a093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:11.408701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1150/2316] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:11.416302 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf4b19-938b-452d-8ae7-299f4ae550cd None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:11.416596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf4b19-938b-452d-8ae7-299f4ae550cd None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:11.416879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf4b19-938b-452d-8ae7-299f4ae550cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:11.437667 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f3bf4b19-938b-452d-8ae7-299f4ae550cd None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:11.439435 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f3bf4b19-938b-452d-8ae7-299f4ae550cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:11.447465 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1167/2317] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:11.452008 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67863e1-60de-4e66-8dc5-ba9ccf96a093 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:11.464176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67863e1-60de-4e66-8dc5-ba9ccf96a093 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:11.469882 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f2f6ca-4598-40a7-b574-a6de3bad9244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:11.470648 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f2f6ca-4598-40a7-b574-a6de3bad9244 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:11.472547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f2f6ca-4598-40a7-b574-a6de3bad9244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:11.472547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f2f6ca-4598-40a7-b574-a6de3bad9244 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:11.472547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f2f6ca-4598-40a7-b574-a6de3bad9244 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:55:11.472547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f2f6ca-4598-40a7-b574-a6de3bad9244 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:11.474306 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f2f6ca-4598-40a7-b574-a6de3bad9244 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:11.487026 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1151/2318] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:55:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:55:11.491616 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf4b19-938b-452d-8ae7-299f4ae550cd None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:11.507340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf4b19-938b-452d-8ae7-299f4ae550cd None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:11.507340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9a248-604a-4a04-bde9-22e6aaf2c12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:11.507340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9a248-604a-4a04-bde9-22e6aaf2c12b admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:11.507689 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9a248-604a-4a04-bde9-22e6aaf2c12b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:11.507947 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9a248-604a-4a04-bde9-22e6aaf2c12b admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:11.508620 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9a248-604a-4a04-bde9-22e6aaf2c12b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:11.510539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9a248-604a-4a04-bde9-22e6aaf2c12b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:11.783373 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1168/2319] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:55:11 2022] POST /identity/v3/projects => generated 381 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:55:11.794034 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f2f6ca-4598-40a7-b574-a6de3bad9244 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:11.804700 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f2f6ca-4598-40a7-b574-a6de3bad9244 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:11.870657 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90cf7d-b36a-4f2b-af01-b522b95e01bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:11.871342 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90cf7d-b36a-4f2b-af01-b522b95e01bd admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:11.871672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90cf7d-b36a-4f2b-af01-b522b95e01bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:11.871950 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90cf7d-b36a-4f2b-af01-b522b95e01bd admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:11.872632 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a90cf7d-b36a-4f2b-af01-b522b95e01bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:11.875861 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a90cf7d-b36a-4f2b-af01-b522b95e01bd admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:12.108295 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1152/2320] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:55:11 2022] POST /identity/v3/users => generated 291 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:55:12.112118 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9a248-604a-4a04-bde9-22e6aaf2c12b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:12.131249 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9a248-604a-4a04-bde9-22e6aaf2c12b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:12.133936 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f1c93-075b-4dbd-a0f6-4d8bd3ef9c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:12.134719 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f1c93-075b-4dbd-a0f6-4d8bd3ef9c3e admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:12.135071 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f1c93-075b-4dbd-a0f6-4d8bd3ef9c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:12.135535 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f1c93-075b-4dbd-a0f6-4d8bd3ef9c3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:12.136086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207f1c93-075b-4dbd-a0f6-4d8bd3ef9c3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:12.137783 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207f1c93-075b-4dbd-a0f6-4d8bd3ef9c3e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:12.157970 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1169/2321] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:55:12.164278 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90cf7d-b36a-4f2b-af01-b522b95e01bd admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:12.174977 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90cf7d-b36a-4f2b-af01-b522b95e01bd admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:12.180794 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d968e-c6b6-4107-8392-98391e1aa308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:12.181264 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d968e-c6b6-4107-8392-98391e1aa308 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:12.181605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d968e-c6b6-4107-8392-98391e1aa308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:12.181958 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d968e-c6b6-4107-8392-98391e1aa308 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:12.182826 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d968e-c6b6-4107-8392-98391e1aa308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:12.185087 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d968e-c6b6-4107-8392-98391e1aa308 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:12.197373 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1153/2322] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:55:12.205648 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f1c93-075b-4dbd-a0f6-4d8bd3ef9c3e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:12.215832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f1c93-075b-4dbd-a0f6-4d8bd3ef9c3e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:12.220881 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c2036-a043-456f-8c99-03aae2794f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:12.221563 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c2036-a043-456f-8c99-03aae2794f9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:12.221850 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c2036-a043-456f-8c99-03aae2794f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:12.222130 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c2036-a043-456f-8c99-03aae2794f9d admin admin] PATH_INFO: `/v3/projects/94a8cfcaa42449ed89b7d4179e899588/users/8c4140f12cc54cd893fff86ce7fccc22/roles/e033961a9363456a8510c34414dd839e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:12.237147 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c2036-a043-456f-8c99-03aae2794f9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=94a8cfcaa42449ed89b7d4179e899588, user_id=8c4140f12cc54cd893fff86ce7fccc22, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:12.239921 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c2036-a043-456f-8c99-03aae2794f9d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:12.290016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afd54a4-417d-42b3-bcee-a1ae4ba38a20 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:12.290302 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afd54a4-417d-42b3-bcee-a1ae4ba38a20 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:12.290578 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afd54a4-417d-42b3-bcee-a1ae4ba38a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:12.310207 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-1afd54a4-417d-42b3-bcee-a1ae4ba38a20 None None] MFA Rules not processed for user `5179c3c4eaaa41f3beb79ba7f1993478`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:12.311480 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1afd54a4-417d-42b3-bcee-a1ae4ba38a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:12.366916 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1154/2323] 10.222.0.97 () {58 vars in 1009 bytes} [Tue Jul 26 14:55:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:55:12.380461 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afd54a4-417d-42b3-bcee-a1ae4ba38a20 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:12.381921 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1afd54a4-417d-42b3-bcee-a1ae4ba38a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:12.405334 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1170/2324] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:55:12 2022] PUT /identity/v3/projects/94a8cfcaa42449ed89b7d4179e899588/users/8c4140f12cc54cd893fff86ce7fccc22/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:12.416042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a5a9b-0f67-435b-8a0a-aa7a8b692367 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:12.416437 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a5a9b-0f67-435b-8a0a-aa7a8b692367 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:12.416843 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a5a9b-0f67-435b-8a0a-aa7a8b692367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:12.451149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-201a5a9b-0f67-435b-8a0a-aa7a8b692367 None None] MFA Rules not processed for user `8c4140f12cc54cd893fff86ce7fccc22`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:12.452708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-201a5a9b-0f67-435b-8a0a-aa7a8b692367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:12.458273 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afd54a4-417d-42b3-bcee-a1ae4ba38a20 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:12.525633 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f269009-8d50-4ff5-bb88-93e3c10ab8aa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85122c4052fd4ba19a2dae185a707caf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:12.526393 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f269009-8d50-4ff5-bb88-93e3c10ab8aa service swift] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:12.526762 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f269009-8d50-4ff5-bb88-93e3c10ab8aa service swift] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:12.527362 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1171/2325] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:12.530659 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f269009-8d50-4ff5-bb88-93e3c10ab8aa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:12.531587 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a5a9b-0f67-435b-8a0a-aa7a8b692367 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:12.595463 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f269009-8d50-4ff5-bb88-93e3c10ab8aa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:12.597103 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f269009-8d50-4ff5-bb88-93e3c10ab8aa service swift] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:12.603910 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a5a9b-0f67-435b-8a0a-aa7a8b692367 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:12.606522 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1155/2326] 10.222.0.97 () {60 vars in 1384 bytes} [Tue Jul 26 14:55:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 14:55:12.609522 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80ba16-8066-41d6-8886-e5358492cde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:12.611207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80ba16-8066-41d6-8886-e5358492cde1 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:12.611668 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80ba16-8066-41d6-8886-e5358492cde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:12.612034 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80ba16-8066-41d6-8886-e5358492cde1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:12.613023 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d80ba16-8066-41d6-8886-e5358492cde1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:12.615222 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d80ba16-8066-41d6-8886-e5358492cde1 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:12.724287 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:12.851311 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1172/2327] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:55:12 2022] POST /identity/v3/projects => generated 381 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:55:12.856102 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f269009-8d50-4ff5-bb88-93e3c10ab8aa service swift] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:12.869640 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f269009-8d50-4ff5-bb88-93e3c10ab8aa service swift] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:12.929057 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cde7b8-14ec-4859-8d5a-bad3ff0e9c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:12.929687 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cde7b8-14ec-4859-8d5a-bad3ff0e9c82 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:12.929973 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cde7b8-14ec-4859-8d5a-bad3ff0e9c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:12.930247 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cde7b8-14ec-4859-8d5a-bad3ff0e9c82 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:12.930943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cde7b8-14ec-4859-8d5a-bad3ff0e9c82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:12.932934 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cde7b8-14ec-4859-8d5a-bad3ff0e9c82 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:13.158659 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1156/2328] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:55:12 2022] POST /identity/v3/users => generated 297 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:55:13.164702 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80ba16-8066-41d6-8886-e5358492cde1 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:13.174372 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80ba16-8066-41d6-8886-e5358492cde1 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:13.180469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfaad87-c361-4baf-85bb-9b77b2ac1362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:13.181206 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfaad87-c361-4baf-85bb-9b77b2ac1362 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:13.181609 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfaad87-c361-4baf-85bb-9b77b2ac1362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:13.181981 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfaad87-c361-4baf-85bb-9b77b2ac1362 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:13.183178 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfaad87-c361-4baf-85bb-9b77b2ac1362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:13.184995 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfaad87-c361-4baf-85bb-9b77b2ac1362 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:13.198796 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1173/2329] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:55:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:55:13.211045 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cde7b8-14ec-4859-8d5a-bad3ff0e9c82 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:13.222291 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cde7b8-14ec-4859-8d5a-bad3ff0e9c82 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:13.230522 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c327a6-c62a-4c33-b48e-a21a60551690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:13.231530 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c327a6-c62a-4c33-b48e-a21a60551690 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:13.232102 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c327a6-c62a-4c33-b48e-a21a60551690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:13.232604 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c327a6-c62a-4c33-b48e-a21a60551690 admin admin] PATH_INFO: `/v3/projects/5afe9502faf34da387060d4c0305488b/users/257951a2ffdf451b8af3001ac405531d/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:13.285460 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c327a6-c62a-4c33-b48e-a21a60551690 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5afe9502faf34da387060d4c0305488b, user_id=257951a2ffdf451b8af3001ac405531d, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:13.290225 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c327a6-c62a-4c33-b48e-a21a60551690 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:13.425240 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1157/2330] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:55:13 2022] PUT /identity/v3/projects/5afe9502faf34da387060d4c0305488b/users/257951a2ffdf451b8af3001ac405531d/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:13.426451 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfaad87-c361-4baf-85bb-9b77b2ac1362 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:13.448831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfaad87-c361-4baf-85bb-9b77b2ac1362 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:13.517622 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2bcecc-26d7-48a9-a8a8-4ba645263a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:13.518375 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2bcecc-26d7-48a9-a8a8-4ba645263a30 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:13.518731 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2bcecc-26d7-48a9-a8a8-4ba645263a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:13.519075 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2bcecc-26d7-48a9-a8a8-4ba645263a30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:13.519975 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2bcecc-26d7-48a9-a8a8-4ba645263a30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:13.521672 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2bcecc-26d7-48a9-a8a8-4ba645263a30 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:13.537001 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1174/2331] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:55:13 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:55:13.541926 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c327a6-c62a-4c33-b48e-a21a60551690 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:13.556408 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c327a6-c62a-4c33-b48e-a21a60551690 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:13.569143 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc73578-5a71-4571-b586-59da08cf9e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:13.570257 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc73578-5a71-4571-b586-59da08cf9e69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:13.570961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc73578-5a71-4571-b586-59da08cf9e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:13.571256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc73578-5a71-4571-b586-59da08cf9e69 admin admin] PATH_INFO: `/v3/projects/5afe9502faf34da387060d4c0305488b/users/257951a2ffdf451b8af3001ac405531d/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:13.574170 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc73578-5a71-4571-b586-59da08cf9e69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5afe9502faf34da387060d4c0305488b, user_id=257951a2ffdf451b8af3001ac405531d, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:13.577290 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc73578-5a71-4571-b586-59da08cf9e69 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:13.608356 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1158/2332] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:55:13 2022] PUT /identity/v3/projects/5afe9502faf34da387060d4c0305488b/users/257951a2ffdf451b8af3001ac405531d/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:13.655131 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0bd94-a849-4b53-84d3-3de793127beb None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:13.655485 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0bd94-a849-4b53-84d3-3de793127beb None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:13.655735 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0bd94-a849-4b53-84d3-3de793127beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:13.676983 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d3303-4065-44f2-84db-ba539d742f64 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:13.677279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d3303-4065-44f2-84db-ba539d742f64 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:13.677578 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d3303-4065-44f2-84db-ba539d742f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:13.710799 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d6c0bd94-a849-4b53-84d3-3de793127beb None None] MFA Rules not processed for user `5179c3c4eaaa41f3beb79ba7f1993478`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:13.710799 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d6c0bd94-a849-4b53-84d3-3de793127beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:13.717685 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6c1d3303-4065-44f2-84db-ba539d742f64 None None] MFA Rules not processed for user `257951a2ffdf451b8af3001ac405531d`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:13.719923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6c1d3303-4065-44f2-84db-ba539d742f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:13.781024 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1175/2333] 10.222.0.97 () {58 vars in 967 bytes} [Tue Jul 26 14:55:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:55:13.800345 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0bd94-a849-4b53-84d3-3de793127beb None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:13.824032 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1159/2334] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:13.837352 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d3303-4065-44f2-84db-ba539d742f64 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:13.838534 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6c1d3303-4065-44f2-84db-ba539d742f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:13.876900 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0bd94-a849-4b53-84d3-3de793127beb None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:13.882273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7157e-10f2-429c-8170-09c60704bd93 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85122c4052fd4ba19a2dae185a707caf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:13.882908 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7157e-10f2-429c-8170-09c60704bd93 service swift] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:13.883194 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7157e-10f2-429c-8170-09c60704bd93 service swift] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:13.883544 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7157e-10f2-429c-8170-09c60704bd93 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:13.893481 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7157e-10f2-429c-8170-09c60704bd93 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:13.895088 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7157e-10f2-429c-8170-09c60704bd93 service swift] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:13.902514 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d3303-4065-44f2-84db-ba539d742f64 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:13.903367 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1176/2335] 10.222.0.97 () {60 vars in 1384 bytes} [Tue Jul 26 14:55:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 14:55:13.910093 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43626741-6140-4398-ac07-3588172995d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:13.910842 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43626741-6140-4398-ac07-3588172995d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:13.911159 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43626741-6140-4398-ac07-3588172995d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:13.911423 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43626741-6140-4398-ac07-3588172995d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:13.919203 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:13.928506 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43626741-6140-4398-ac07-3588172995d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:13.930750 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43626741-6140-4398-ac07-3588172995d4 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:13.941525 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1160/2336] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:13 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:55:14.133815 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7157e-10f2-429c-8170-09c60704bd93 service swift] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:14.143356 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7157e-10f2-429c-8170-09c60704bd93 service swift] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:14.147146 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a90e42-e62d-439b-ae33-0c9dc317b5b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:14.147972 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a90e42-e62d-439b-ae33-0c9dc317b5b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:14.154600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a90e42-e62d-439b-ae33-0c9dc317b5b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:14.154600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a90e42-e62d-439b-ae33-0c9dc317b5b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:14.205617 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a90e42-e62d-439b-ae33-0c9dc317b5b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:14.207061 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a90e42-e62d-439b-ae33-0c9dc317b5b4 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:14.216638 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1177/2337] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:55:15.299769 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43626741-6140-4398-ac07-3588172995d4 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:15.367868 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43626741-6140-4398-ac07-3588172995d4 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:15.373030 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab88bc3-db64-4261-a91f-f05d88902d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:15.373688 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab88bc3-db64-4261-a91f-f05d88902d77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:15.373973 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab88bc3-db64-4261-a91f-f05d88902d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:15.374252 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab88bc3-db64-4261-a91f-f05d88902d77 admin admin] PATH_INFO: `/v3/users/8c4140f12cc54cd893fff86ce7fccc22` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:15.375791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab88bc3-db64-4261-a91f-f05d88902d77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c4140f12cc54cd893fff86ce7fccc22)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:15.377785 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab88bc3-db64-4261-a91f-f05d88902d77 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:15.691255 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3ab88bc3-db64-4261-a91f-f05d88902d77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c4140f12cc54cd893fff86ce7fccc22'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:15.691255 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3ab88bc3-db64-4261-a91f-f05d88902d77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c4140f12cc54cd893fff86ce7fccc22'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:15.696701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3ab88bc3-db64-4261-a91f-f05d88902d77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c4140f12cc54cd893fff86ce7fccc22'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:15.709327 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3ab88bc3-db64-4261-a91f-f05d88902d77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c4140f12cc54cd893fff86ce7fccc22'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:15.710054 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3ab88bc3-db64-4261-a91f-f05d88902d77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c4140f12cc54cd893fff86ce7fccc22'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:15.821637 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1161/2338] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:55:15 2022] DELETE /identity/v3/users/8c4140f12cc54cd893fff86ce7fccc22 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:15.826622 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbce8f-039f-4892-917a-9765226599b9 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:15.826622 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbce8f-039f-4892-917a-9765226599b9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:15.826622 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbce8f-039f-4892-917a-9765226599b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:15.839030 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:15.850688 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-bebbce8f-039f-4892-917a-9765226599b9 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:15.851943 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-bebbce8f-039f-4892-917a-9765226599b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:15.913226 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1178/2339] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:15.923129 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab88bc3-db64-4261-a91f-f05d88902d77 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:15.924584 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3ab88bc3-db64-4261-a91f-f05d88902d77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:15.981921 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab88bc3-db64-4261-a91f-f05d88902d77 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:15.985740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903528ba-cccf-4c35-a388-04a815724693 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:15.986358 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903528ba-cccf-4c35-a388-04a815724693 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:15.986640 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903528ba-cccf-4c35-a388-04a815724693 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:15.986915 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903528ba-cccf-4c35-a388-04a815724693 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:15.995897 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903528ba-cccf-4c35-a388-04a815724693 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:15.997335 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903528ba-cccf-4c35-a388-04a815724693 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:16.007107 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1162/2340] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:55:16.531390 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebbce8f-039f-4892-917a-9765226599b9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:16.533131 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-bebbce8f-039f-4892-917a-9765226599b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:16.549159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebbce8f-039f-4892-917a-9765226599b9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:16.554617 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9139524b-bb53-4d83-941a-0272c1ecdbca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:16.555176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139524b-bb53-4d83-941a-0272c1ecdbca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:16.555485 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139524b-bb53-4d83-941a-0272c1ecdbca admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:16.555769 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139524b-bb53-4d83-941a-0272c1ecdbca admin admin] PATH_INFO: `/v3/projects/94a8cfcaa42449ed89b7d4179e899588` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:16.557095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9139524b-bb53-4d83-941a-0272c1ecdbca admin admin] RBAC: Authorizing `identity:delete_project(project_id=94a8cfcaa42449ed89b7d4179e899588)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:16.559054 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9139524b-bb53-4d83-941a-0272c1ecdbca admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:16.776539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9139524b-bb53-4d83-941a-0272c1ecdbca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94a8cfcaa42449ed89b7d4179e899588', 'request_id': 'req-9139524b-bb53-4d83-941a-0272c1ecdbca'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:16.781980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9139524b-bb53-4d83-941a-0272c1ecdbca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94a8cfcaa42449ed89b7d4179e899588', 'request_id': 'req-9139524b-bb53-4d83-941a-0272c1ecdbca'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:16.873063 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9139524b-bb53-4d83-941a-0272c1ecdbca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94a8cfcaa42449ed89b7d4179e899588', 'request_id': 'req-9139524b-bb53-4d83-941a-0272c1ecdbca'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:16.888471 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9139524b-bb53-4d83-941a-0272c1ecdbca admin admin] The token cache is being invalidate because project 94a8cfcaa42449ed89b7d4179e899588 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:55:16.889015 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9139524b-bb53-4d83-941a-0272c1ecdbca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:16.889665 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9139524b-bb53-4d83-941a-0272c1ecdbca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:16.891504 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1179/2341] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:55:16 2022] DELETE /identity/v3/projects/94a8cfcaa42449ed89b7d4179e899588 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:16.895588 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903528ba-cccf-4c35-a388-04a815724693 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:16.905188 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903528ba-cccf-4c35-a388-04a815724693 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:16.961166 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dda75ad-68d0-454b-bde4-0cecd768deb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:16.961803 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda75ad-68d0-454b-bde4-0cecd768deb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:16.962084 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda75ad-68d0-454b-bde4-0cecd768deb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:16.962433 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda75ad-68d0-454b-bde4-0cecd768deb1 admin admin] PATH_INFO: `/v3/users/257951a2ffdf451b8af3001ac405531d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:16.963638 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dda75ad-68d0-454b-bde4-0cecd768deb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=257951a2ffdf451b8af3001ac405531d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:16.965566 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dda75ad-68d0-454b-bde4-0cecd768deb1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:16.998789 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:17.148646 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-7dda75ad-68d0-454b-bde4-0cecd768deb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '257951a2ffdf451b8af3001ac405531d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:17.149299 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-7dda75ad-68d0-454b-bde4-0cecd768deb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '257951a2ffdf451b8af3001ac405531d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:17.161587 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-7dda75ad-68d0-454b-bde4-0cecd768deb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '257951a2ffdf451b8af3001ac405531d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:17.168320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-7dda75ad-68d0-454b-bde4-0cecd768deb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '257951a2ffdf451b8af3001ac405531d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:17.172222 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-7dda75ad-68d0-454b-bde4-0cecd768deb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '257951a2ffdf451b8af3001ac405531d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:17.284458 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1163/2342] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:55:16 2022] DELETE /identity/v3/users/257951a2ffdf451b8af3001ac405531d => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:17.713013 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:17.760556 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9139524b-bb53-4d83-941a-0272c1ecdbca admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:17.781209 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9139524b-bb53-4d83-941a-0272c1ecdbca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:17.781209 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9139524b-bb53-4d83-941a-0272c1ecdbca admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:17.841734 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95019af-f962-496b-8b08-1db48c1ae182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:17.842298 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95019af-f962-496b-8b08-1db48c1ae182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:17.842586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95019af-f962-496b-8b08-1db48c1ae182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:17.842899 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95019af-f962-496b-8b08-1db48c1ae182 admin admin] PATH_INFO: `/v3/projects/5afe9502faf34da387060d4c0305488b` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:17.844061 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95019af-f962-496b-8b08-1db48c1ae182 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5afe9502faf34da387060d4c0305488b)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:17.845994 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95019af-f962-496b-8b08-1db48c1ae182 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:18.016921 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e95019af-f962-496b-8b08-1db48c1ae182 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5afe9502faf34da387060d4c0305488b', 'request_id': 'req-e95019af-f962-496b-8b08-1db48c1ae182'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:18.022358 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e95019af-f962-496b-8b08-1db48c1ae182 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5afe9502faf34da387060d4c0305488b', 'request_id': 'req-e95019af-f962-496b-8b08-1db48c1ae182'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:18.124750 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:18.152384 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e95019af-f962-496b-8b08-1db48c1ae182 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5afe9502faf34da387060d4c0305488b', 'request_id': 'req-e95019af-f962-496b-8b08-1db48c1ae182'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:18.163855 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e95019af-f962-496b-8b08-1db48c1ae182 admin admin] The token cache is being invalidate because project 5afe9502faf34da387060d4c0305488b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:55:18.164146 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e95019af-f962-496b-8b08-1db48c1ae182 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:18.165787 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e95019af-f962-496b-8b08-1db48c1ae182 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:18.166407 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1180/2343] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:55:17 2022] DELETE /identity/v3/projects/5afe9502faf34da387060d4c0305488b => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:18.171281 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac1fff-10a9-4ac9-ba6e-72e14cc95e59 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:18.171756 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac1fff-10a9-4ac9-ba6e-72e14cc95e59 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:18.171935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac1fff-10a9-4ac9-ba6e-72e14cc95e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:18.191643 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b4ac1fff-10a9-4ac9-ba6e-72e14cc95e59 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:18.193677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b4ac1fff-10a9-4ac9-ba6e-72e14cc95e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:18.252011 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1164/2344] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:18.259947 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884af43a-67a0-4376-a48b-d852179b9833 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:18.260055 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884af43a-67a0-4376-a48b-d852179b9833 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:18.260300 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884af43a-67a0-4376-a48b-d852179b9833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:18.273245 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:18.282921 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-884af43a-67a0-4376-a48b-d852179b9833 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:18.284256 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-884af43a-67a0-4376-a48b-d852179b9833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:18.292332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1181/2345] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:18.296546 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac1fff-10a9-4ac9-ba6e-72e14cc95e59 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:18.306689 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac1fff-10a9-4ac9-ba6e-72e14cc95e59 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:18.307649 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:18.311543 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79de82b-49d3-41a9-bfb8-150198350c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:18.312190 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79de82b-49d3-41a9-bfb8-150198350c06 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:18.312508 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79de82b-49d3-41a9-bfb8-150198350c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:18.312790 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79de82b-49d3-41a9-bfb8-150198350c06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:18.314147 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79de82b-49d3-41a9-bfb8-150198350c06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:55:18.314453 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79de82b-49d3-41a9-bfb8-150198350c06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:18.316225 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79de82b-49d3-41a9-bfb8-150198350c06 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:18.328156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1165/2346] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:55:18.332442 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884af43a-67a0-4376-a48b-d852179b9833 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:18.341563 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884af43a-67a0-4376-a48b-d852179b9833 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:18.346275 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6417a39-a8af-41f9-8b70-80238812dda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:18.346889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6417a39-a8af-41f9-8b70-80238812dda4 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:18.347175 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6417a39-a8af-41f9-8b70-80238812dda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:18.347519 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6417a39-a8af-41f9-8b70-80238812dda4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:18.348191 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6417a39-a8af-41f9-8b70-80238812dda4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:18.350091 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6417a39-a8af-41f9-8b70-80238812dda4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:18.490301 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1182/2347] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:55:18 2022] POST /identity/v3/projects => generated 377 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:55:18.491702 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79de82b-49d3-41a9-bfb8-150198350c06 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:18.502147 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79de82b-49d3-41a9-bfb8-150198350c06 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:18.559744 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce7ba5-093b-4616-befa-bcb79185b9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:18.560364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce7ba5-093b-4616-befa-bcb79185b9fd admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:18.560653 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce7ba5-093b-4616-befa-bcb79185b9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:18.560932 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce7ba5-093b-4616-befa-bcb79185b9fd admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:18.562200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce7ba5-093b-4616-befa-bcb79185b9fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:18.564157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce7ba5-093b-4616-befa-bcb79185b9fd admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:18.615461 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:18.678437 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1166/2348] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:55:18 2022] POST /identity/v3/users => generated 289 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:55:18.682506 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6417a39-a8af-41f9-8b70-80238812dda4 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:18.694790 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6417a39-a8af-41f9-8b70-80238812dda4 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:18.699634 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649453e7-c320-4658-b605-22855d5c7c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:18.700323 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649453e7-c320-4658-b605-22855d5c7c58 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:18.700811 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649453e7-c320-4658-b605-22855d5c7c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:18.701330 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649453e7-c320-4658-b605-22855d5c7c58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:18.702052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649453e7-c320-4658-b605-22855d5c7c58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:18.705259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649453e7-c320-4658-b605-22855d5c7c58 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:18.715148 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1183/2349] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:55:18.719162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce7ba5-093b-4616-befa-bcb79185b9fd admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:18.729324 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce7ba5-093b-4616-befa-bcb79185b9fd admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:18.733976 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd2790-4b3e-4a21-a64a-3c805a826546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:18.734609 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd2790-4b3e-4a21-a64a-3c805a826546 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:18.735097 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd2790-4b3e-4a21-a64a-3c805a826546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:18.735654 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd2790-4b3e-4a21-a64a-3c805a826546 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:18.736735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd2790-4b3e-4a21-a64a-3c805a826546 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:18.738415 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd2790-4b3e-4a21-a64a-3c805a826546 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:18.749213 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1167/2350] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:55:18.753776 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649453e7-c320-4658-b605-22855d5c7c58 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:18.763890 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649453e7-c320-4658-b605-22855d5c7c58 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:18.769560 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d030a152-2943-44da-a16f-2dffdfe117b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:18.770230 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030a152-2943-44da-a16f-2dffdfe117b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:18.770515 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030a152-2943-44da-a16f-2dffdfe117b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:18.770794 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030a152-2943-44da-a16f-2dffdfe117b1 admin admin] PATH_INFO: `/v3/projects/4705017825cc46139b39449c17f6c14f/users/9afe7746763e448aaeb604741245850c/roles/e033961a9363456a8510c34414dd839e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:18.789913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d030a152-2943-44da-a16f-2dffdfe117b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4705017825cc46139b39449c17f6c14f, user_id=9afe7746763e448aaeb604741245850c, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:18.792709 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d030a152-2943-44da-a16f-2dffdfe117b1 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:18.794169 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:18.873210 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1184/2351] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:55:18 2022] PUT /identity/v3/projects/4705017825cc46139b39449c17f6c14f/users/9afe7746763e448aaeb604741245850c/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:18.885267 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86363d-e17e-422b-a933-9cdb18dad9a5 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:18.885618 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86363d-e17e-422b-a933-9cdb18dad9a5 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:18.885897 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86363d-e17e-422b-a933-9cdb18dad9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:18.913584 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-2a86363d-e17e-422b-a933-9cdb18dad9a5 None None] MFA Rules not processed for user `9afe7746763e448aaeb604741245850c`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:18.919310 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2a86363d-e17e-422b-a933-9cdb18dad9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:18.980373 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1168/2352] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:18.991068 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d030a152-2943-44da-a16f-2dffdfe117b1 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:18.992774 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d030a152-2943-44da-a16f-2dffdfe117b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:19.003313 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:19.051586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d030a152-2943-44da-a16f-2dffdfe117b1 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:19.055376 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9967e9-6b74-45e7-9e8c-44ec8d5d3af0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62adb1f5e9654927847e1717630ff6b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:19.056419 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9967e9-6b74-45e7-9e8c-44ec8d5d3af0 service glance] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:19.056419 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9967e9-6b74-45e7-9e8c-44ec8d5d3af0 service glance] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:19.056643 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9967e9-6b74-45e7-9e8c-44ec8d5d3af0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:19.065672 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9967e9-6b74-45e7-9e8c-44ec8d5d3af0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:19.067103 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9967e9-6b74-45e7-9e8c-44ec8d5d3af0 service glance] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:19.075718 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1185/2353] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:55:20.383736 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a86363d-e17e-422b-a933-9cdb18dad9a5 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:20.385145 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2a86363d-e17e-422b-a933-9cdb18dad9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:20.430543 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:20.452020 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a86363d-e17e-422b-a933-9cdb18dad9a5 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:20.457158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641e455-58c6-4e86-9eae-d52f1db7a1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:20.457799 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641e455-58c6-4e86-9eae-d52f1db7a1e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:20.458111 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641e455-58c6-4e86-9eae-d52f1db7a1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:20.458384 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641e455-58c6-4e86-9eae-d52f1db7a1e5 admin admin] PATH_INFO: `/v3/users/81d4b47691f44cab80c244a0d21542db` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:20.459606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b641e455-58c6-4e86-9eae-d52f1db7a1e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81d4b47691f44cab80c244a0d21542db)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:20.461891 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b641e455-58c6-4e86-9eae-d52f1db7a1e5 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:20.724323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b641e455-58c6-4e86-9eae-d52f1db7a1e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81d4b47691f44cab80c244a0d21542db'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:20.725040 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b641e455-58c6-4e86-9eae-d52f1db7a1e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81d4b47691f44cab80c244a0d21542db'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:20.738387 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b641e455-58c6-4e86-9eae-d52f1db7a1e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81d4b47691f44cab80c244a0d21542db'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:20.745331 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b641e455-58c6-4e86-9eae-d52f1db7a1e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81d4b47691f44cab80c244a0d21542db'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:20.746016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b641e455-58c6-4e86-9eae-d52f1db7a1e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81d4b47691f44cab80c244a0d21542db'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:20.999197 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1169/2354] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:55:20 2022] DELETE /identity/v3/users/81d4b47691f44cab80c244a0d21542db => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:21.011754 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046c3fb-100d-4e02-a6de-444e595fc0b5 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:21.012430 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046c3fb-100d-4e02-a6de-444e595fc0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:21.012430 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046c3fb-100d-4e02-a6de-444e595fc0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:21.035146 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a046c3fb-100d-4e02-a6de-444e595fc0b5 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:21.036321 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a046c3fb-100d-4e02-a6de-444e595fc0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:21.102138 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1186/2355] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:21.111972 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641e455-58c6-4e86-9eae-d52f1db7a1e5 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:21.113402 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b641e455-58c6-4e86-9eae-d52f1db7a1e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:21.183263 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641e455-58c6-4e86-9eae-d52f1db7a1e5 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:21.189070 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0cdd61-2362-4234-85d5-19ce03b15ab0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:21.189710 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cdd61-2362-4234-85d5-19ce03b15ab0 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:21.190019 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cdd61-2362-4234-85d5-19ce03b15ab0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:21.190299 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cdd61-2362-4234-85d5-19ce03b15ab0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:21.200754 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0cdd61-2362-4234-85d5-19ce03b15ab0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:21.202332 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0cdd61-2362-4234-85d5-19ce03b15ab0 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:21.214372 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1170/2356] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:55:21.509193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046c3fb-100d-4e02-a6de-444e595fc0b5 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:21.510650 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a046c3fb-100d-4e02-a6de-444e595fc0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:21.527152 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046c3fb-100d-4e02-a6de-444e595fc0b5 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:21.533793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486a2d83-5bb7-4df4-8b68-c1fe1f849de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:21.534906 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a2d83-5bb7-4df4-8b68-c1fe1f849de3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:21.535469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a2d83-5bb7-4df4-8b68-c1fe1f849de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:21.535963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a2d83-5bb7-4df4-8b68-c1fe1f849de3 admin admin] PATH_INFO: `/v3/projects/bd4e47dfa8ff43829da20aa31c4bb25b` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:21.537337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486a2d83-5bb7-4df4-8b68-c1fe1f849de3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd4e47dfa8ff43829da20aa31c4bb25b)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:21.542171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486a2d83-5bb7-4df4-8b68-c1fe1f849de3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:21.707452 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-486a2d83-5bb7-4df4-8b68-c1fe1f849de3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd4e47dfa8ff43829da20aa31c4bb25b', 'request_id': 'req-486a2d83-5bb7-4df4-8b68-c1fe1f849de3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:21.713226 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-486a2d83-5bb7-4df4-8b68-c1fe1f849de3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd4e47dfa8ff43829da20aa31c4bb25b', 'request_id': 'req-486a2d83-5bb7-4df4-8b68-c1fe1f849de3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:21.810232 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-486a2d83-5bb7-4df4-8b68-c1fe1f849de3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd4e47dfa8ff43829da20aa31c4bb25b', 'request_id': 'req-486a2d83-5bb7-4df4-8b68-c1fe1f849de3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:21.823126 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-486a2d83-5bb7-4df4-8b68-c1fe1f849de3 admin admin] The token cache is being invalidate because project bd4e47dfa8ff43829da20aa31c4bb25b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:55:21.823450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-486a2d83-5bb7-4df4-8b68-c1fe1f849de3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:21.839165 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-486a2d83-5bb7-4df4-8b68-c1fe1f849de3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:21.839165 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1187/2357] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:55:21 2022] DELETE /identity/v3/projects/bd4e47dfa8ff43829da20aa31c4bb25b => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:21.839386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a67a2b-fdcd-4a51-8c07-75f632d3b53e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:21.839386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a67a2b-fdcd-4a51-8c07-75f632d3b53e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:21.839386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a67a2b-fdcd-4a51-8c07-75f632d3b53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:21.851495 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-d0a67a2b-fdcd-4a51-8c07-75f632d3b53e None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:21.852697 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d0a67a2b-fdcd-4a51-8c07-75f632d3b53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:21.916259 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1171/2358] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:21.925843 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81108e1-6187-41e7-9814-6bdbe274e522 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:21.926140 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81108e1-6187-41e7-9814-6bdbe274e522 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:21.926420 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81108e1-6187-41e7-9814-6bdbe274e522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:21.948052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d81108e1-6187-41e7-9814-6bdbe274e522 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:21.949275 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d81108e1-6187-41e7-9814-6bdbe274e522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:21.958162 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1188/2359] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:21.962452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a67a2b-fdcd-4a51-8c07-75f632d3b53e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:21.968736 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:21.972621 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a67a2b-fdcd-4a51-8c07-75f632d3b53e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:21.977456 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc3c84-d042-4326-bf44-acd59cde96b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:21.978082 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc3c84-d042-4326-bf44-acd59cde96b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:21.978366 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc3c84-d042-4326-bf44-acd59cde96b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:21.978645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc3c84-d042-4326-bf44-acd59cde96b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:21.979372 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cc3c84-d042-4326-bf44-acd59cde96b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:55:21.979723 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cc3c84-d042-4326-bf44-acd59cde96b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:21.981390 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cc3c84-d042-4326-bf44-acd59cde96b8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:21.996647 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1172/2360] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:55:21.998879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81108e1-6187-41e7-9814-6bdbe274e522 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:22.008755 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81108e1-6187-41e7-9814-6bdbe274e522 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:22.013776 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f81150-348d-4ee6-89a8-f568de85f844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:22.014396 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f81150-348d-4ee6-89a8-f568de85f844 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:22.014690 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f81150-348d-4ee6-89a8-f568de85f844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:22.014974 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f81150-348d-4ee6-89a8-f568de85f844 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:22.015708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f81150-348d-4ee6-89a8-f568de85f844 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:22.018582 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f81150-348d-4ee6-89a8-f568de85f844 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:22.133489 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1189/2361] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:55:21 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:55:22.137199 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc3c84-d042-4326-bf44-acd59cde96b8 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:22.147805 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc3c84-d042-4326-bf44-acd59cde96b8 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:22.206454 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895482b7-b0f7-4199-b49c-8d8a106e730f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:22.207184 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895482b7-b0f7-4199-b49c-8d8a106e730f admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:22.207499 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895482b7-b0f7-4199-b49c-8d8a106e730f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:22.207779 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895482b7-b0f7-4199-b49c-8d8a106e730f admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:22.208515 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895482b7-b0f7-4199-b49c-8d8a106e730f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:22.210527 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895482b7-b0f7-4199-b49c-8d8a106e730f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:22.335303 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1173/2362] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:55:22 2022] POST /identity/v3/users => generated 287 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:55:22.336737 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f81150-348d-4ee6-89a8-f568de85f844 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:22.353556 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:22.357814 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f81150-348d-4ee6-89a8-f568de85f844 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:22.363310 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a92a7c0-bb0d-45ba-8eb1-6afa3b982073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:22.363964 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92a7c0-bb0d-45ba-8eb1-6afa3b982073 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:22.364253 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92a7c0-bb0d-45ba-8eb1-6afa3b982073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:22.364533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92a7c0-bb0d-45ba-8eb1-6afa3b982073 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:22.365273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a92a7c0-bb0d-45ba-8eb1-6afa3b982073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:22.366877 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a92a7c0-bb0d-45ba-8eb1-6afa3b982073 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:22.377732 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1190/2363] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:55:22.382037 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895482b7-b0f7-4199-b49c-8d8a106e730f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:22.392368 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895482b7-b0f7-4199-b49c-8d8a106e730f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:22.397106 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223539ea-ab30-40e3-97a6-360efe50ae59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:22.405625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223539ea-ab30-40e3-97a6-360efe50ae59 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:22.405625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223539ea-ab30-40e3-97a6-360efe50ae59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:22.405625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223539ea-ab30-40e3-97a6-360efe50ae59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:22.405625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223539ea-ab30-40e3-97a6-360efe50ae59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:22.405625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223539ea-ab30-40e3-97a6-360efe50ae59 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:22.411521 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1174/2364] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:55:22.415636 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a92a7c0-bb0d-45ba-8eb1-6afa3b982073 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:22.426579 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a92a7c0-bb0d-45ba-8eb1-6afa3b982073 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:22.431252 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1851d-d576-4deb-9631-7e594f2203ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:22.435011 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1851d-d576-4deb-9631-7e594f2203ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:22.435011 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1851d-d576-4deb-9631-7e594f2203ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:22.435011 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1851d-d576-4deb-9631-7e594f2203ab admin admin] PATH_INFO: `/v3/projects/fde004be36ee4efeaa9ada32b4e2182c/users/29225fc9f22f4ec7b2979334760fc5ea/roles/e033961a9363456a8510c34414dd839e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:22.447898 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1851d-d576-4deb-9631-7e594f2203ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=fde004be36ee4efeaa9ada32b4e2182c, user_id=29225fc9f22f4ec7b2979334760fc5ea, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:22.450783 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1851d-d576-4deb-9631-7e594f2203ab admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:22.672949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1191/2365] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:55:22 2022] PUT /identity/v3/projects/fde004be36ee4efeaa9ada32b4e2182c/users/29225fc9f22f4ec7b2979334760fc5ea/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:22.678275 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b7f65-5dfe-4502-8fd7-78242e12b2a2 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:22.678620 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b7f65-5dfe-4502-8fd7-78242e12b2a2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:22.678871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b7f65-5dfe-4502-8fd7-78242e12b2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:22.699000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-742b7f65-5dfe-4502-8fd7-78242e12b2a2 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:22.700427 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-742b7f65-5dfe-4502-8fd7-78242e12b2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:22.758378 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1175/2366] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:22.769412 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1851d-d576-4deb-9631-7e594f2203ab admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:22.771534 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b2a1851d-d576-4deb-9631-7e594f2203ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:22.834879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1851d-d576-4deb-9631-7e594f2203ab admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:22.839056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c6fe69-e530-40c0-9d57-6a238b69cfce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:22.839748 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6fe69-e530-40c0-9d57-6a238b69cfce service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:22.840039 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6fe69-e530-40c0-9d57-6a238b69cfce service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:22.840498 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6fe69-e530-40c0-9d57-6a238b69cfce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:22.850355 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c6fe69-e530-40c0-9d57-6a238b69cfce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:22.851881 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c6fe69-e530-40c0-9d57-6a238b69cfce service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:22.862121 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1192/2367] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:55:23.380677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b7f65-5dfe-4502-8fd7-78242e12b2a2 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:23.382281 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-742b7f65-5dfe-4502-8fd7-78242e12b2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:23.394800 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b7f65-5dfe-4502-8fd7-78242e12b2a2 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:23.399424 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f888c2-73eb-4895-aaa8-891bc0c9cd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:23.400103 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f888c2-73eb-4895-aaa8-891bc0c9cd4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:23.400337 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f888c2-73eb-4895-aaa8-891bc0c9cd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:23.400674 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f888c2-73eb-4895-aaa8-891bc0c9cd4e admin admin] PATH_INFO: `/v3/users/973ce853a0894326894cba4eb44813a0` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:23.401760 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f888c2-73eb-4895-aaa8-891bc0c9cd4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=973ce853a0894326894cba4eb44813a0)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:23.403754 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f888c2-73eb-4895-aaa8-891bc0c9cd4e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:23.659456 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b0f888c2-73eb-4895-aaa8-891bc0c9cd4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '973ce853a0894326894cba4eb44813a0'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:23.660236 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b0f888c2-73eb-4895-aaa8-891bc0c9cd4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '973ce853a0894326894cba4eb44813a0'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:23.672122 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b0f888c2-73eb-4895-aaa8-891bc0c9cd4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '973ce853a0894326894cba4eb44813a0'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:23.673846 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:23.680155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b0f888c2-73eb-4895-aaa8-891bc0c9cd4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '973ce853a0894326894cba4eb44813a0'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:23.680820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b0f888c2-73eb-4895-aaa8-891bc0c9cd4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '973ce853a0894326894cba4eb44813a0'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:23.849960 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1176/2368] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:55:23 2022] DELETE /identity/v3/users/973ce853a0894326894cba4eb44813a0 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:23.854911 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fb02a-30fa-4f97-874e-60593ddef2e7 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:23.855312 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fb02a-30fa-4f97-874e-60593ddef2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:23.855764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fb02a-30fa-4f97-874e-60593ddef2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:23.877578 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6b8fb02a-30fa-4f97-874e-60593ddef2e7 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:23.878978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6b8fb02a-30fa-4f97-874e-60593ddef2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:23.940718 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1193/2369] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:23.950516 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f888c2-73eb-4895-aaa8-891bc0c9cd4e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:23.951976 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b0f888c2-73eb-4895-aaa8-891bc0c9cd4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:24.011770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f888c2-73eb-4895-aaa8-891bc0c9cd4e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:24.015446 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfcfcc3-f34e-4a91-9d44-75024b6e7344 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:24.016355 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcfcc3-f34e-4a91-9d44-75024b6e7344 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:24.016636 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcfcc3-f34e-4a91-9d44-75024b6e7344 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:24.019453 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcfcc3-f34e-4a91-9d44-75024b6e7344 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:24.026773 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfcfcc3-f34e-4a91-9d44-75024b6e7344 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:24.028454 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfcfcc3-f34e-4a91-9d44-75024b6e7344 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:24.039924 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1177/2370] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:55:24.481116 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8fb02a-30fa-4f97-874e-60593ddef2e7 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:24.483062 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6b8fb02a-30fa-4f97-874e-60593ddef2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:24.496832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8fb02a-30fa-4f97-874e-60593ddef2e7 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:24.503604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe642e-7442-4789-bc6f-7e09a8954373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:24.504358 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe642e-7442-4789-bc6f-7e09a8954373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:24.505033 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe642e-7442-4789-bc6f-7e09a8954373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:24.505419 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe642e-7442-4789-bc6f-7e09a8954373 admin admin] PATH_INFO: `/v3/projects/32d3753b5f524ab78fd777c9c9c7b9ed` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:24.507025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe642e-7442-4789-bc6f-7e09a8954373 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32d3753b5f524ab78fd777c9c9c7b9ed)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:24.509005 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe642e-7442-4789-bc6f-7e09a8954373 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:24.725397 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2ffe642e-7442-4789-bc6f-7e09a8954373 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32d3753b5f524ab78fd777c9c9c7b9ed', 'request_id': 'req-2ffe642e-7442-4789-bc6f-7e09a8954373'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:24.731641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2ffe642e-7442-4789-bc6f-7e09a8954373 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32d3753b5f524ab78fd777c9c9c7b9ed', 'request_id': 'req-2ffe642e-7442-4789-bc6f-7e09a8954373'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:24.839604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2ffe642e-7442-4789-bc6f-7e09a8954373 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32d3753b5f524ab78fd777c9c9c7b9ed', 'request_id': 'req-2ffe642e-7442-4789-bc6f-7e09a8954373'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:24.843289 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:24.854092 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2ffe642e-7442-4789-bc6f-7e09a8954373 admin admin] The token cache is being invalidate because project 32d3753b5f524ab78fd777c9c9c7b9ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:55:24.854618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2ffe642e-7442-4789-bc6f-7e09a8954373 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:24.855216 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2ffe642e-7442-4789-bc6f-7e09a8954373 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:24.856836 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1194/2371] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:55:24 2022] DELETE /identity/v3/projects/32d3753b5f524ab78fd777c9c9c7b9ed => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:24.861173 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfcfcc3-f34e-4a91-9d44-75024b6e7344 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:24.871586 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfcfcc3-f34e-4a91-9d44-75024b6e7344 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:24.949278 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fcf0f7-a271-4840-a09b-ad876e0542dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:24.949579 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcf0f7-a271-4840-a09b-ad876e0542dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:24.949899 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcf0f7-a271-4840-a09b-ad876e0542dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:24.950230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcf0f7-a271-4840-a09b-ad876e0542dc admin admin] PATH_INFO: `/v3/users/a88acee527f140fbaccc8fae9c221902` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:24.951522 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fcf0f7-a271-4840-a09b-ad876e0542dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a88acee527f140fbaccc8fae9c221902)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:24.953531 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fcf0f7-a271-4840-a09b-ad876e0542dc admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:25.121894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-08fcf0f7-a271-4840-a09b-ad876e0542dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a88acee527f140fbaccc8fae9c221902'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:25.122659 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-08fcf0f7-a271-4840-a09b-ad876e0542dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a88acee527f140fbaccc8fae9c221902'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:25.140644 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-08fcf0f7-a271-4840-a09b-ad876e0542dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a88acee527f140fbaccc8fae9c221902'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:25.145405 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-08fcf0f7-a271-4840-a09b-ad876e0542dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a88acee527f140fbaccc8fae9c221902'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:25.146218 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-08fcf0f7-a271-4840-a09b-ad876e0542dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a88acee527f140fbaccc8fae9c221902'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:25.230930 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1178/2372] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:55:24 2022] DELETE /identity/v3/users/a88acee527f140fbaccc8fae9c221902 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:25.817165 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe642e-7442-4789-bc6f-7e09a8954373 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:25.818598 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2ffe642e-7442-4789-bc6f-7e09a8954373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:25.831512 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe642e-7442-4789-bc6f-7e09a8954373 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:25.901705 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c1270f-0289-4717-92ca-f97cf2d6b90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:25.902455 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1270f-0289-4717-92ca-f97cf2d6b90e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:25.902811 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1270f-0289-4717-92ca-f97cf2d6b90e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:25.903157 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1270f-0289-4717-92ca-f97cf2d6b90e admin admin] PATH_INFO: `/v3/projects/095ce9a40fa04425998d94e3d5d91624` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:25.904476 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c1270f-0289-4717-92ca-f97cf2d6b90e admin admin] RBAC: Authorizing `identity:delete_project(project_id=095ce9a40fa04425998d94e3d5d91624)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:25.906407 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c1270f-0289-4717-92ca-f97cf2d6b90e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:26.106529 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f9c1270f-0289-4717-92ca-f97cf2d6b90e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '095ce9a40fa04425998d94e3d5d91624', 'request_id': 'req-f9c1270f-0289-4717-92ca-f97cf2d6b90e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:26.113414 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f9c1270f-0289-4717-92ca-f97cf2d6b90e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '095ce9a40fa04425998d94e3d5d91624', 'request_id': 'req-f9c1270f-0289-4717-92ca-f97cf2d6b90e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:26.222181 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f9c1270f-0289-4717-92ca-f97cf2d6b90e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '095ce9a40fa04425998d94e3d5d91624', 'request_id': 'req-f9c1270f-0289-4717-92ca-f97cf2d6b90e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:26.239439 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f9c1270f-0289-4717-92ca-f97cf2d6b90e admin admin] The token cache is being invalidate because project 095ce9a40fa04425998d94e3d5d91624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:55:26.239741 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f9c1270f-0289-4717-92ca-f97cf2d6b90e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:26.240442 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f9c1270f-0289-4717-92ca-f97cf2d6b90e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:26.242539 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1195/2373] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:55:25 2022] DELETE /identity/v3/projects/095ce9a40fa04425998d94e3d5d91624 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:26.248693 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec1945-b66f-4741-9d77-a031412221dc None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:26.249229 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec1945-b66f-4741-9d77-a031412221dc None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:26.249229 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec1945-b66f-4741-9d77-a031412221dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:26.261355 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:26.287288 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-aaec1945-b66f-4741-9d77-a031412221dc None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:26.289178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-aaec1945-b66f-4741-9d77-a031412221dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:26.319637 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c1270f-0289-4717-92ca-f97cf2d6b90e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:26.326795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f9c1270f-0289-4717-92ca-f97cf2d6b90e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:26.380445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1179/2374] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:26.417885 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c1270f-0289-4717-92ca-f97cf2d6b90e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:26.424309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f8892-67af-448d-978d-a0ae1658a228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:26.427249 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f8892-67af-448d-978d-a0ae1658a228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:26.430791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f8892-67af-448d-978d-a0ae1658a228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:26.430791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f8892-67af-448d-978d-a0ae1658a228 admin admin] PATH_INFO: `/v3/users/b5dfeaec44c84504901041aefb801f93` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:26.430791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f8892-67af-448d-978d-a0ae1658a228 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5dfeaec44c84504901041aefb801f93)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:26.431945 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f8892-67af-448d-978d-a0ae1658a228 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:26.455367 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf91a7a-7bd7-4af7-be6d-22b87036d814 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:26.455698 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf91a7a-7bd7-4af7-be6d-22b87036d814 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:26.455986 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf91a7a-7bd7-4af7-be6d-22b87036d814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:26.482815 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9cf91a7a-7bd7-4af7-be6d-22b87036d814 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:26.484194 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9cf91a7a-7bd7-4af7-be6d-22b87036d814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:26.507989 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1180/2375] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:55:26.511484 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf91a7a-7bd7-4af7-be6d-22b87036d814 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:26.535951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf91a7a-7bd7-4af7-be6d-22b87036d814 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:26.542410 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b38d9be-a4ec-4b25-b3fa-1ba28db6c91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:26.543140 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38d9be-a4ec-4b25-b3fa-1ba28db6c91d admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:26.543526 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38d9be-a4ec-4b25-b3fa-1ba28db6c91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:26.543876 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38d9be-a4ec-4b25-b3fa-1ba28db6c91d admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:26.544692 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b38d9be-a4ec-4b25-b3fa-1ba28db6c91d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:55:26.545028 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b38d9be-a4ec-4b25-b3fa-1ba28db6c91d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:26.546720 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b38d9be-a4ec-4b25-b3fa-1ba28db6c91d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:26.557179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:26.575356 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1181/2376] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:55:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:55:26.580879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b38d9be-a4ec-4b25-b3fa-1ba28db6c91d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:26.605985 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b38d9be-a4ec-4b25-b3fa-1ba28db6c91d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:26.613783 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0978b62-1658-400a-a909-8b88613cb457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:26.614660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0978b62-1658-400a-a909-8b88613cb457 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:26.614660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0978b62-1658-400a-a909-8b88613cb457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:26.614927 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0978b62-1658-400a-a909-8b88613cb457 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:26.615802 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0978b62-1658-400a-a909-8b88613cb457 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:26.617881 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0978b62-1658-400a-a909-8b88613cb457 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:26.749571 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7b3f8892-67af-448d-978d-a0ae1658a228 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5dfeaec44c84504901041aefb801f93'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:26.750259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7b3f8892-67af-448d-978d-a0ae1658a228 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5dfeaec44c84504901041aefb801f93'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:26.764240 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7b3f8892-67af-448d-978d-a0ae1658a228 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5dfeaec44c84504901041aefb801f93'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:26.781359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7b3f8892-67af-448d-978d-a0ae1658a228 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5dfeaec44c84504901041aefb801f93'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:26.783914 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7b3f8892-67af-448d-978d-a0ae1658a228 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5dfeaec44c84504901041aefb801f93'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:26.798804 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1182/2377] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:55:26 2022] POST /identity/v3/projects => generated 385 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:55:26.799866 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0978b62-1658-400a-a909-8b88613cb457 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:26.801590 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c0978b62-1658-400a-a909-8b88613cb457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:26.897055 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0978b62-1658-400a-a909-8b88613cb457 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:26.903087 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c5afe-794c-4b8d-a585-bb1f0a4a6d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:26.903973 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c5afe-794c-4b8d-a585-bb1f0a4a6d74 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:26.904848 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c5afe-794c-4b8d-a585-bb1f0a4a6d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:26.905333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c5afe-794c-4b8d-a585-bb1f0a4a6d74 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:26.906177 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c5afe-794c-4b8d-a585-bb1f0a4a6d74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:26.909107 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c5afe-794c-4b8d-a585-bb1f0a4a6d74 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:26.913843 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1196/2378] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:55:26 2022] DELETE /identity/v3/users/b5dfeaec44c84504901041aefb801f93 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:55:26.921182 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e3a76-8b55-44ac-9cd7-f0d7fb81c3c7 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:26.921570 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e3a76-8b55-44ac-9cd7-f0d7fb81c3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:26.921926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e3a76-8b55-44ac-9cd7-f0d7fb81c3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:26.943962 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-023e3a76-8b55-44ac-9cd7-f0d7fb81c3c7 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:26.944973 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-023e3a76-8b55-44ac-9cd7-f0d7fb81c3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:27.014075 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1197/2379] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:27.036685 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e3a76-8b55-44ac-9cd7-f0d7fb81c3c7 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:27.038231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-023e3a76-8b55-44ac-9cd7-f0d7fb81c3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:27.044045 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1183/2380] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:55:26 2022] POST /identity/v3/users => generated 293 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:55:27.045276 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c5afe-794c-4b8d-a585-bb1f0a4a6d74 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:27.057256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c5afe-794c-4b8d-a585-bb1f0a4a6d74 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:27.062896 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c600b-de75-4269-aac7-d81de4ddea96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:27.064485 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c600b-de75-4269-aac7-d81de4ddea96 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:27.065016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c600b-de75-4269-aac7-d81de4ddea96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:27.065504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c600b-de75-4269-aac7-d81de4ddea96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:27.066553 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c600b-de75-4269-aac7-d81de4ddea96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:27.068632 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c600b-de75-4269-aac7-d81de4ddea96 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:27.103317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1184/2381] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:55:27.108888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c600b-de75-4269-aac7-d81de4ddea96 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:27.119272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c600b-de75-4269-aac7-d81de4ddea96 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:27.124380 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ed9b3-2aea-4bf1-b17e-4a9234eb4ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:27.126573 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ed9b3-2aea-4bf1-b17e-4a9234eb4ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:27.126573 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ed9b3-2aea-4bf1-b17e-4a9234eb4ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:27.126573 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ed9b3-2aea-4bf1-b17e-4a9234eb4ef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:27.127023 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ed9b3-2aea-4bf1-b17e-4a9234eb4ef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:27.128735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ed9b3-2aea-4bf1-b17e-4a9234eb4ef7 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:27.146690 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1185/2382] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:55:27.147644 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ed9b3-2aea-4bf1-b17e-4a9234eb4ef7 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:27.158053 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e3a76-8b55-44ac-9cd7-f0d7fb81c3c7 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:27.158771 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ed9b3-2aea-4bf1-b17e-4a9234eb4ef7 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:27.162753 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14205496-af00-4930-9862-6d308a972e75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:27.166158 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14205496-af00-4930-9862-6d308a972e75 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:27.166278 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e0e42-527e-4071-a89f-4c472ee54a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:27.166389 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14205496-af00-4930-9862-6d308a972e75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:27.166389 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14205496-af00-4930-9862-6d308a972e75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:27.166552 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e0e42-527e-4071-a89f-4c472ee54a75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:27.166552 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e0e42-527e-4071-a89f-4c472ee54a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:27.166552 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e0e42-527e-4071-a89f-4c472ee54a75 admin admin] PATH_INFO: `/v3/projects/04bd0428330c4ca39e3b6cf6083fabfd/users/c946c5cbfe444916abe2560eaa9ced59/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:27.176618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14205496-af00-4930-9862-6d308a972e75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:27.178772 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14205496-af00-4930-9862-6d308a972e75 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:27.182093 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e0e42-527e-4071-a89f-4c472ee54a75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04bd0428330c4ca39e3b6cf6083fabfd, user_id=c946c5cbfe444916abe2560eaa9ced59, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:27.187632 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e0e42-527e-4071-a89f-4c472ee54a75 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:27.192759 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1198/2383] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:55:27.276716 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1186/2384] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:55:27 2022] PUT /identity/v3/projects/04bd0428330c4ca39e3b6cf6083fabfd/users/c946c5cbfe444916abe2560eaa9ced59/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:27.279207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05065e-f609-4843-a4c2-f58953eb8880 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:27.279596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05065e-f609-4843-a4c2-f58953eb8880 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:27.280702 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05065e-f609-4843-a4c2-f58953eb8880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:27.325430 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0b05065e-f609-4843-a4c2-f58953eb8880 None None] MFA Rules not processed for user `c946c5cbfe444916abe2560eaa9ced59`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:27.326662 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0b05065e-f609-4843-a4c2-f58953eb8880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:27.431725 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1199/2385] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:27.439508 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e0e42-527e-4071-a89f-4c472ee54a75 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:27.441608 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ad5e0e42-527e-4071-a89f-4c472ee54a75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:27.536943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e0e42-527e-4071-a89f-4c472ee54a75 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:27.546216 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c6c087-56ee-49e4-a0a8-67a40a6a9936 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:27.547088 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6c087-56ee-49e4-a0a8-67a40a6a9936 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:27.547761 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6c087-56ee-49e4-a0a8-67a40a6a9936 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:27.548387 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6c087-56ee-49e4-a0a8-67a40a6a9936 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:27.558485 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c6c087-56ee-49e4-a0a8-67a40a6a9936 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:27.560317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c6c087-56ee-49e4-a0a8-67a40a6a9936 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:27.570944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1187/2386] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:55:27.596086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b05065e-f609-4843-a4c2-f58953eb8880 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:27.686455 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b05065e-f609-4843-a4c2-f58953eb8880 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:27.693400 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d0de5-7764-49d6-acd6-6911940e4ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:27.694454 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d0de5-7764-49d6-acd6-6911940e4ccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:27.694636 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d0de5-7764-49d6-acd6-6911940e4ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:27.695020 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d0de5-7764-49d6-acd6-6911940e4ccc admin admin] PATH_INFO: `/v3/users/c946c5cbfe444916abe2560eaa9ced59` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:27.696770 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6d0de5-7764-49d6-acd6-6911940e4ccc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c946c5cbfe444916abe2560eaa9ced59)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:27.699667 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6d0de5-7764-49d6-acd6-6911940e4ccc admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:27.881693 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c6c087-56ee-49e4-a0a8-67a40a6a9936 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:27.883338 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-76c6c087-56ee-49e4-a0a8-67a40a6a9936 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:27.902234 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c6c087-56ee-49e4-a0a8-67a40a6a9936 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:27.907164 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354dbc28-5155-4b34-951b-61b21fe490c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:27.908159 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354dbc28-5155-4b34-951b-61b21fe490c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:27.908511 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354dbc28-5155-4b34-951b-61b21fe490c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:27.908862 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354dbc28-5155-4b34-951b-61b21fe490c3 admin admin] PATH_INFO: `/v3/projects/8d9966e52d2e4bc8a0f5659f2f494104` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:27.910232 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354dbc28-5155-4b34-951b-61b21fe490c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d9966e52d2e4bc8a0f5659f2f494104)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:27.912280 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354dbc28-5155-4b34-951b-61b21fe490c3 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:27.969847 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fb6d0de5-7764-49d6-acd6-6911940e4ccc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c946c5cbfe444916abe2560eaa9ced59'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:27.970580 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fb6d0de5-7764-49d6-acd6-6911940e4ccc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c946c5cbfe444916abe2560eaa9ced59'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:27.987282 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fb6d0de5-7764-49d6-acd6-6911940e4ccc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c946c5cbfe444916abe2560eaa9ced59'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:27.993405 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fb6d0de5-7764-49d6-acd6-6911940e4ccc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c946c5cbfe444916abe2560eaa9ced59'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:27.994259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fb6d0de5-7764-49d6-acd6-6911940e4ccc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c946c5cbfe444916abe2560eaa9ced59'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:28.080639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-354dbc28-5155-4b34-951b-61b21fe490c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d9966e52d2e4bc8a0f5659f2f494104', 'request_id': 'req-354dbc28-5155-4b34-951b-61b21fe490c3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:28.087428 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-354dbc28-5155-4b34-951b-61b21fe490c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d9966e52d2e4bc8a0f5659f2f494104', 'request_id': 'req-354dbc28-5155-4b34-951b-61b21fe490c3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:28.117509 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1200/2387] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:55:27 2022] DELETE /identity/v3/users/c946c5cbfe444916abe2560eaa9ced59 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:28.122056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944f2cf-cbac-45a3-9379-9470c577fe2e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:28.122357 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944f2cf-cbac-45a3-9379-9470c577fe2e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:28.122764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944f2cf-cbac-45a3-9379-9470c577fe2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:28.146806 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-8944f2cf-cbac-45a3-9379-9470c577fe2e None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:28.148608 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8944f2cf-cbac-45a3-9379-9470c577fe2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:28.192824 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-354dbc28-5155-4b34-951b-61b21fe490c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d9966e52d2e4bc8a0f5659f2f494104', 'request_id': 'req-354dbc28-5155-4b34-951b-61b21fe490c3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:28.208621 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-354dbc28-5155-4b34-951b-61b21fe490c3 admin admin] The token cache is being invalidate because project 8d9966e52d2e4bc8a0f5659f2f494104 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:55:28.209241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-354dbc28-5155-4b34-951b-61b21fe490c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:28.226529 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-354dbc28-5155-4b34-951b-61b21fe490c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:28.230719 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1188/2388] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:55:27 2022] DELETE /identity/v3/projects/8d9966e52d2e4bc8a0f5659f2f494104 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:55:28.242272 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1201/2389] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:55:28.243785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944f2cf-cbac-45a3-9379-9470c577fe2e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:28.245397 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619fd51e-021a-4e07-8ad6-1f120cccd13b None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:28.245692 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619fd51e-021a-4e07-8ad6-1f120cccd13b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:28.245872 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8944f2cf-cbac-45a3-9379-9470c577fe2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:28.245981 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619fd51e-021a-4e07-8ad6-1f120cccd13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:28.272164 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-619fd51e-021a-4e07-8ad6-1f120cccd13b None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:28.273686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-619fd51e-021a-4e07-8ad6-1f120cccd13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:28.292362 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1189/2390] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:28.293888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f1581-8b68-44fa-8416-c64208b67ebe None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:28.294243 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f1581-8b68-44fa-8416-c64208b67ebe None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:28.294566 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f1581-8b68-44fa-8416-c64208b67ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:28.315321 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-099f1581-8b68-44fa-8416-c64208b67ebe None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:28.316596 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-099f1581-8b68-44fa-8416-c64208b67ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:28.321588 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944f2cf-cbac-45a3-9379-9470c577fe2e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:28.326605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:28.327549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1190/2391] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:28.328523 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ece0da-589a-48bc-a624-32099947dc09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:28.329696 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ece0da-589a-48bc-a624-32099947dc09 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:28.329979 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ece0da-589a-48bc-a624-32099947dc09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:28.330276 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ece0da-589a-48bc-a624-32099947dc09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:28.331482 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099f1581-8b68-44fa-8416-c64208b67ebe None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:28.332889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d9ece0da-589a-48bc-a624-32099947dc09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:28.342392 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099f1581-8b68-44fa-8416-c64208b67ebe None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:28.350287 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61600530-fcd8-447c-9b1e-1d2a6d93f5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:28.350474 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ece0da-589a-48bc-a624-32099947dc09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:28.351026 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61600530-fcd8-447c-9b1e-1d2a6d93f5f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:28.351626 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61600530-fcd8-447c-9b1e-1d2a6d93f5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:28.352089 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61600530-fcd8-447c-9b1e-1d2a6d93f5f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:28.352912 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61600530-fcd8-447c-9b1e-1d2a6d93f5f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:55:28.353223 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61600530-fcd8-447c-9b1e-1d2a6d93f5f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:28.356997 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ece0da-589a-48bc-a624-32099947dc09 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:28.357115 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61600530-fcd8-447c-9b1e-1d2a6d93f5f6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:28.365825 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1202/2392] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:55:28.375583 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1191/2393] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:55:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:55:28.378091 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ece0da-589a-48bc-a624-32099947dc09 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:28.393312 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ece0da-589a-48bc-a624-32099947dc09 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:28.402492 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb40a5e1-982a-4163-b96e-3c1297fe12a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:28.403237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb40a5e1-982a-4163-b96e-3c1297fe12a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:28.403634 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb40a5e1-982a-4163-b96e-3c1297fe12a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:28.404134 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb40a5e1-982a-4163-b96e-3c1297fe12a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:28.406051 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb40a5e1-982a-4163-b96e-3c1297fe12a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:28.408523 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb40a5e1-982a-4163-b96e-3c1297fe12a6 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:28.545731 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1203/2394] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:55:28 2022] POST /identity/v3/projects => generated 365 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:55:28.549730 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61600530-fcd8-447c-9b1e-1d2a6d93f5f6 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:28.566229 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:28.567935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61600530-fcd8-447c-9b1e-1d2a6d93f5f6 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:28.656695 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db673cf-fc0c-49e3-9d03-45fd920acd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:28.657936 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db673cf-fc0c-49e3-9d03-45fd920acd94 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:28.659902 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db673cf-fc0c-49e3-9d03-45fd920acd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:28.659902 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db673cf-fc0c-49e3-9d03-45fd920acd94 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:28.660462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db673cf-fc0c-49e3-9d03-45fd920acd94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:28.662938 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db673cf-fc0c-49e3-9d03-45fd920acd94 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:28.789856 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1192/2395] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:55:28 2022] POST /identity/v3/users => generated 283 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:55:28.793655 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb40a5e1-982a-4163-b96e-3c1297fe12a6 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:28.809809 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb40a5e1-982a-4163-b96e-3c1297fe12a6 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:28.818870 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed4dc3-7fd0-4fb2-b5e2-80129e027848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:28.819656 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed4dc3-7fd0-4fb2-b5e2-80129e027848 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:28.820141 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed4dc3-7fd0-4fb2-b5e2-80129e027848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:28.820305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed4dc3-7fd0-4fb2-b5e2-80129e027848 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:28.821839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed4dc3-7fd0-4fb2-b5e2-80129e027848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:28.822939 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed4dc3-7fd0-4fb2-b5e2-80129e027848 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:28.835438 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1204/2396] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:55:28.839572 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db673cf-fc0c-49e3-9d03-45fd920acd94 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:28.850590 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db673cf-fc0c-49e3-9d03-45fd920acd94 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:28.857969 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f0645-d58d-44b0-81b6-fbde86369141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:28.857969 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f0645-d58d-44b0-81b6-fbde86369141 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:28.857969 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f0645-d58d-44b0-81b6-fbde86369141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:28.857969 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f0645-d58d-44b0-81b6-fbde86369141 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:28.858473 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f0645-d58d-44b0-81b6-fbde86369141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:28.860573 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f0645-d58d-44b0-81b6-fbde86369141 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:28.885452 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1193/2397] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:55:28.890569 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed4dc3-7fd0-4fb2-b5e2-80129e027848 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:28.910091 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed4dc3-7fd0-4fb2-b5e2-80129e027848 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:28.919031 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f0645-d58d-44b0-81b6-fbde86369141 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:28.920726 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b93f0645-d58d-44b0-81b6-fbde86369141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:28.925854 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b53cf1-83a6-41eb-b01a-3a864e78b2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:28.926687 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b53cf1-83a6-41eb-b01a-3a864e78b2fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:28.926871 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b53cf1-83a6-41eb-b01a-3a864e78b2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:28.927123 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b53cf1-83a6-41eb-b01a-3a864e78b2fc admin admin] PATH_INFO: `/v3/projects/9b6cc17049d04b6c81762ebb834ca2e2/users/06c447b856644b65aa2d07b44d10c871/roles/e033961a9363456a8510c34414dd839e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:28.948493 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f0645-d58d-44b0-81b6-fbde86369141 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:28.966298 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3ae55-98f7-4626-9954-555c6ce86e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:28.967127 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3ae55-98f7-4626-9954-555c6ce86e72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:28.967791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3ae55-98f7-4626-9954-555c6ce86e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:28.968043 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3ae55-98f7-4626-9954-555c6ce86e72 admin admin] PATH_INFO: `/v3/projects/04bd0428330c4ca39e3b6cf6083fabfd` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:28.969774 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc3ae55-98f7-4626-9954-555c6ce86e72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04bd0428330c4ca39e3b6cf6083fabfd)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:28.969954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b53cf1-83a6-41eb-b01a-3a864e78b2fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b6cc17049d04b6c81762ebb834ca2e2, user_id=06c447b856644b65aa2d07b44d10c871, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:28.972580 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc3ae55-98f7-4626-9954-555c6ce86e72 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:28.973337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b53cf1-83a6-41eb-b01a-3a864e78b2fc admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:29.035232 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1205/2398] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:55:28 2022] PUT /identity/v3/projects/9b6cc17049d04b6c81762ebb834ca2e2/users/06c447b856644b65aa2d07b44d10c871/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:29.045625 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf8ff8-72af-4676-bba8-e27d17039f1e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:29.045935 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf8ff8-72af-4676-bba8-e27d17039f1e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:29.046184 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf8ff8-72af-4676-bba8-e27d17039f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:29.077856 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-90bf8ff8-72af-4676-bba8-e27d17039f1e None None] MFA Rules not processed for user `06c447b856644b65aa2d07b44d10c871`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:29.079844 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-90bf8ff8-72af-4676-bba8-e27d17039f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:29.153095 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-afc3ae55-98f7-4626-9954-555c6ce86e72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04bd0428330c4ca39e3b6cf6083fabfd', 'request_id': 'req-afc3ae55-98f7-4626-9954-555c6ce86e72'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:29.163138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-afc3ae55-98f7-4626-9954-555c6ce86e72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04bd0428330c4ca39e3b6cf6083fabfd', 'request_id': 'req-afc3ae55-98f7-4626-9954-555c6ce86e72'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:29.176947 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1206/2399] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:29 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:55:29.190552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf8ff8-72af-4676-bba8-e27d17039f1e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:29.198995 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-90bf8ff8-72af-4676-bba8-e27d17039f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:29.250546 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-afc3ae55-98f7-4626-9954-555c6ce86e72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04bd0428330c4ca39e3b6cf6083fabfd', 'request_id': 'req-afc3ae55-98f7-4626-9954-555c6ce86e72'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:29.266110 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-afc3ae55-98f7-4626-9954-555c6ce86e72 admin admin] The token cache is being invalidate because project 04bd0428330c4ca39e3b6cf6083fabfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:55:29.267024 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-afc3ae55-98f7-4626-9954-555c6ce86e72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:29.267992 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-afc3ae55-98f7-4626-9954-555c6ce86e72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:29.271087 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1194/2400] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:55:28 2022] DELETE /identity/v3/projects/04bd0428330c4ca39e3b6cf6083fabfd => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:55:29.276953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1dca1-34bd-457a-8bde-7d1bcaac36b9 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:29.279718 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1dca1-34bd-457a-8bde-7d1bcaac36b9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:29.280293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1dca1-34bd-457a-8bde-7d1bcaac36b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:29.312476 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf8ff8-72af-4676-bba8-e27d17039f1e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:29.316680 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a1b1dca1-34bd-457a-8bde-7d1bcaac36b9 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:29.318495 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a1b1dca1-34bd-457a-8bde-7d1bcaac36b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:29.362518 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:29.405627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e245f5-857d-457d-9727-442c2f3842d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:29.407514 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e245f5-857d-457d-9727-442c2f3842d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:29.407514 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e245f5-857d-457d-9727-442c2f3842d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:29.407514 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e245f5-857d-457d-9727-442c2f3842d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:29.409716 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-87e245f5-857d-457d-9727-442c2f3842d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:29.410033 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1195/2401] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:29.419081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88fc4d9-d09d-4785-b3eb-47be6b1e67a1 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:29.419667 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88fc4d9-d09d-4785-b3eb-47be6b1e67a1 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:29.420317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88fc4d9-d09d-4785-b3eb-47be6b1e67a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:29.444067 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-d88fc4d9-d09d-4785-b3eb-47be6b1e67a1 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:29.445124 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d88fc4d9-d09d-4785-b3eb-47be6b1e67a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:29.453617 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1196/2402] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:29.455003 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722f32a-b2f2-41f0-90d0-96adb44cc5c5 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:29.455290 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722f32a-b2f2-41f0-90d0-96adb44cc5c5 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:29.455650 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722f32a-b2f2-41f0-90d0-96adb44cc5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:29.488942 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-2722f32a-b2f2-41f0-90d0-96adb44cc5c5 None None] MFA Rules not processed for user `29225fc9f22f4ec7b2979334760fc5ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:29.490170 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2722f32a-b2f2-41f0-90d0-96adb44cc5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:29.506257 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e245f5-857d-457d-9727-442c2f3842d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:29.507752 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e245f5-857d-457d-9727-442c2f3842d9 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:29.521014 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1207/2403] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:29 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:55:29.521014 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e245f5-857d-457d-9727-442c2f3842d9 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:29.532373 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e245f5-857d-457d-9727-442c2f3842d9 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:29.542930 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea5962-c04a-4b83-8a97-6cc4840fa4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:29.543678 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea5962-c04a-4b83-8a97-6cc4840fa4ac admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:29.543969 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea5962-c04a-4b83-8a97-6cc4840fa4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:29.544249 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea5962-c04a-4b83-8a97-6cc4840fa4ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:29.549763 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea5962-c04a-4b83-8a97-6cc4840fa4ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:55:29.549763 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea5962-c04a-4b83-8a97-6cc4840fa4ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:29.551698 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea5962-c04a-4b83-8a97-6cc4840fa4ac admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:29.575100 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1208/2404] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:55:29.580822 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea5962-c04a-4b83-8a97-6cc4840fa4ac admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:29.584212 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1197/2405] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:29.590247 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722f32a-b2f2-41f0-90d0-96adb44cc5c5 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:29.592508 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea5962-c04a-4b83-8a97-6cc4840fa4ac admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:29.592985 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2722f32a-b2f2-41f0-90d0-96adb44cc5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:29.598017 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e816723-a863-46b4-b4e7-550e3b4cf27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:29.599128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e816723-a863-46b4-b4e7-550e3b4cf27d admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:29.599578 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e816723-a863-46b4-b4e7-550e3b4cf27d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:29.599966 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e816723-a863-46b4-b4e7-550e3b4cf27d admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:29.600751 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e816723-a863-46b4-b4e7-550e3b4cf27d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:29.602925 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e816723-a863-46b4-b4e7-550e3b4cf27d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:29.609731 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722f32a-b2f2-41f0-90d0-96adb44cc5c5 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:29.620154 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2c2e1-f686-4003-ac93-98c5afb4d645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:29.620792 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2c2e1-f686-4003-ac93-98c5afb4d645 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:29.621201 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2c2e1-f686-4003-ac93-98c5afb4d645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:29.621334 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2c2e1-f686-4003-ac93-98c5afb4d645 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:29.626472 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b2c2e1-f686-4003-ac93-98c5afb4d645 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:29.626918 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b2c2e1-f686-4003-ac93-98c5afb4d645 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:29.694245 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1209/2406] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:55:29 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:55:29.698544 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e816723-a863-46b4-b4e7-550e3b4cf27d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:29.731428 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:29.751655 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1198/2407] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:55:29 2022] POST /identity/v3/projects => generated 371 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:55:29.754241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2c2e1-f686-4003-ac93-98c5afb4d645 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:29.776596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e816723-a863-46b4-b4e7-550e3b4cf27d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:29.824946 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2c2e1-f686-4003-ac93-98c5afb4d645 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:29.837915 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a028b95-97a9-4539-b754-e6a266c28032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:29.838870 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a028b95-97a9-4539-b754-e6a266c28032 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:29.839835 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a028b95-97a9-4539-b754-e6a266c28032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:29.840462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a028b95-97a9-4539-b754-e6a266c28032 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:29.841177 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a028b95-97a9-4539-b754-e6a266c28032 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:29.843429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a028b95-97a9-4539-b754-e6a266c28032 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:29.846388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6df1cc5-0713-48fc-bcf2-4f527bc72575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:29.847121 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df1cc5-0713-48fc-bcf2-4f527bc72575 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:29.847513 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df1cc5-0713-48fc-bcf2-4f527bc72575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:29.847860 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df1cc5-0713-48fc-bcf2-4f527bc72575 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:29.848593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6df1cc5-0713-48fc-bcf2-4f527bc72575 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:29.850716 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6df1cc5-0713-48fc-bcf2-4f527bc72575 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:29.968855 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1199/2408] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:55:29 2022] POST /identity/v3/users => generated 292 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:55:29.969849 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a028b95-97a9-4539-b754-e6a266c28032 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:29.970702 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1210/2409] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:55:29 2022] POST /identity/v3/users => generated 293 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:55:29.971586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6df1cc5-0713-48fc-bcf2-4f527bc72575 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:29.980453 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a028b95-97a9-4539-b754-e6a266c28032 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:29.984179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6df1cc5-0713-48fc-bcf2-4f527bc72575 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:29.985484 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f394c-6af4-47e6-bc38-b2e5b02ca4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:29.986246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f394c-6af4-47e6-bc38-b2e5b02ca4e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:29.986599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f394c-6af4-47e6-bc38-b2e5b02ca4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:29.986946 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f394c-6af4-47e6-bc38-b2e5b02ca4e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:29.987850 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f394c-6af4-47e6-bc38-b2e5b02ca4e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:29.989522 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f394c-6af4-47e6-bc38-b2e5b02ca4e0 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:29.990974 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2fd46b-bbff-4d01-bd73-91e50b0bbfb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:29.991833 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2fd46b-bbff-4d01-bd73-91e50b0bbfb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:29.992094 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2fd46b-bbff-4d01-bd73-91e50b0bbfb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:29.992364 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2fd46b-bbff-4d01-bd73-91e50b0bbfb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:29.993055 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2fd46b-bbff-4d01-bd73-91e50b0bbfb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:29.994658 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2fd46b-bbff-4d01-bd73-91e50b0bbfb7 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:30.000636 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1200/2410] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:55:30.005161 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f394c-6af4-47e6-bc38-b2e5b02ca4e0 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:30.005633 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1211/2411] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:55:30.009748 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2fd46b-bbff-4d01-bd73-91e50b0bbfb7 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:30.019356 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f394c-6af4-47e6-bc38-b2e5b02ca4e0 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:30.020173 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2fd46b-bbff-4d01-bd73-91e50b0bbfb7 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:30.024149 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ab6b2-6ab3-4e30-b6e9-387257b511c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:30.024810 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0454a3e-f486-449b-872f-0a8c3b29787b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:30.024943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ab6b2-6ab3-4e30-b6e9-387257b511c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:30.025097 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ab6b2-6ab3-4e30-b6e9-387257b511c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:30.025375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ab6b2-6ab3-4e30-b6e9-387257b511c9 admin admin] PATH_INFO: `/v3/projects/3f7526cec950478aa97ec6dd5b4827f2/users/96a877efdba748489e5a5bb3181d8d47/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:30.025474 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0454a3e-f486-449b-872f-0a8c3b29787b admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:30.025735 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0454a3e-f486-449b-872f-0a8c3b29787b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:30.025966 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0454a3e-f486-449b-872f-0a8c3b29787b admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:30.026652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0454a3e-f486-449b-872f-0a8c3b29787b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:30.028278 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0454a3e-f486-449b-872f-0a8c3b29787b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:30.038793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1212/2412] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:55:30.042894 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0454a3e-f486-449b-872f-0a8c3b29787b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:30.046413 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ab6b2-6ab3-4e30-b6e9-387257b511c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f7526cec950478aa97ec6dd5b4827f2, user_id=96a877efdba748489e5a5bb3181d8d47, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:30.050221 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ab6b2-6ab3-4e30-b6e9-387257b511c9 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:30.055076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0454a3e-f486-449b-872f-0a8c3b29787b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:30.059928 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0cf3d-80b0-4d53-ad3a-e02e17f7a931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:30.064402 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0cf3d-80b0-4d53-ad3a-e02e17f7a931 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:30.064402 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0cf3d-80b0-4d53-ad3a-e02e17f7a931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:30.064402 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0cf3d-80b0-4d53-ad3a-e02e17f7a931 admin admin] PATH_INFO: `/v3/projects/846d2a9d718e4b2ebf48996a639d4299/users/80e71dc1fa894d98bc57195fbf4c415c/roles/e033961a9363456a8510c34414dd839e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:30.078578 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a0cf3d-80b0-4d53-ad3a-e02e17f7a931 admin admin] RBAC: Authorizing `identity:create_grant(project_id=846d2a9d718e4b2ebf48996a639d4299, user_id=80e71dc1fa894d98bc57195fbf4c415c, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:30.081409 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a0cf3d-80b0-4d53-ad3a-e02e17f7a931 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:30.148363 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1201/2413] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:55:30 2022] PUT /identity/v3/projects/3f7526cec950478aa97ec6dd5b4827f2/users/96a877efdba748489e5a5bb3181d8d47/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:55:30.152162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ab6b2-6ab3-4e30-b6e9-387257b511c9 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:30.207452 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1213/2414] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:55:30 2022] PUT /identity/v3/projects/846d2a9d718e4b2ebf48996a639d4299/users/80e71dc1fa894d98bc57195fbf4c415c/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:55:30.209182 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c27d3-7342-4148-a333-cab507bf9be7 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:30.209478 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c27d3-7342-4148-a333-cab507bf9be7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:30.209753 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c27d3-7342-4148-a333-cab507bf9be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:30.224584 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ab6b2-6ab3-4e30-b6e9-387257b511c9 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:30.239284 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-fa8c27d3-7342-4148-a333-cab507bf9be7 None None] MFA Rules not processed for user `80e71dc1fa894d98bc57195fbf4c415c`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:30.240518 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fa8c27d3-7342-4148-a333-cab507bf9be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:30.285382 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6437ecd-4c3f-4981-b34c-02690458e62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:30.286560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6437ecd-4c3f-4981-b34c-02690458e62d admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:30.286950 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6437ecd-4c3f-4981-b34c-02690458e62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:30.287335 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6437ecd-4c3f-4981-b34c-02690458e62d admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:30.288237 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6437ecd-4c3f-4981-b34c-02690458e62d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:30.290141 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6437ecd-4c3f-4981-b34c-02690458e62d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:30.303696 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1202/2415] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:55:30.309590 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6437ecd-4c3f-4981-b34c-02690458e62d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:30.323288 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6437ecd-4c3f-4981-b34c-02690458e62d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:30.329244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf39d3a-cd4e-479d-ba15-7612e3f76a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:30.330065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf39d3a-cd4e-479d-ba15-7612e3f76a4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:30.330482 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf39d3a-cd4e-479d-ba15-7612e3f76a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:30.330824 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf39d3a-cd4e-479d-ba15-7612e3f76a4e admin admin] PATH_INFO: `/v3/projects/3f7526cec950478aa97ec6dd5b4827f2/users/96a877efdba748489e5a5bb3181d8d47/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:30.336340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1214/2416] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:30.336652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf39d3a-cd4e-479d-ba15-7612e3f76a4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f7526cec950478aa97ec6dd5b4827f2, user_id=96a877efdba748489e5a5bb3181d8d47, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:30.336652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf39d3a-cd4e-479d-ba15-7612e3f76a4e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:30.346639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1203/2417] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:55:30 2022] PUT /identity/v3/projects/3f7526cec950478aa97ec6dd5b4827f2/users/96a877efdba748489e5a5bb3181d8d47/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:55:30.349416 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c27d3-7342-4148-a333-cab507bf9be7 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:30.354030 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fa8c27d3-7342-4148-a333-cab507bf9be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:30.431467 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c27d3-7342-4148-a333-cab507bf9be7 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:30.488495 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52e2ff3-8a0f-4307-8c5a-5fe7462cfc4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:30.488495 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e2ff3-8a0f-4307-8c5a-5fe7462cfc4f service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:30.488495 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e2ff3-8a0f-4307-8c5a-5fe7462cfc4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:30.488495 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e2ff3-8a0f-4307-8c5a-5fe7462cfc4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:30.555549 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52e2ff3-8a0f-4307-8c5a-5fe7462cfc4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:30.557249 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52e2ff3-8a0f-4307-8c5a-5fe7462cfc4f service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:30.568856 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1215/2418] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:55:31.256599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:33.005219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:36.654145 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:36.805432 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:37.670368 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823b0f0-2a49-4d9c-bfaa-82461883ad36 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:37.670835 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823b0f0-2a49-4d9c-bfaa-82461883ad36 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:37.671242 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823b0f0-2a49-4d9c-bfaa-82461883ad36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:37.711950 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6823b0f0-2a49-4d9c-bfaa-82461883ad36 None None] MFA Rules not processed for user `96a877efdba748489e5a5bb3181d8d47`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:37.713177 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6823b0f0-2a49-4d9c-bfaa-82461883ad36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:37.784473 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1204/2419] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:37 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:37.804602 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52e2ff3-8a0f-4307-8c5a-5fe7462cfc4f service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:37.804602 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c52e2ff3-8a0f-4307-8c5a-5fe7462cfc4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:37.881616 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52e2ff3-8a0f-4307-8c5a-5fe7462cfc4f service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:37.887479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538277e3-52b2-484f-82b2-fe2268956015 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:37.888105 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538277e3-52b2-484f-82b2-fe2268956015 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:37.888389 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538277e3-52b2-484f-82b2-fe2268956015 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:37.888720 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538277e3-52b2-484f-82b2-fe2268956015 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:37.948757 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538277e3-52b2-484f-82b2-fe2268956015 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:37.950379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538277e3-52b2-484f-82b2-fe2268956015 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:37.959849 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1216/2420] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:55:38.560447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:41.108652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:41.750496 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62365e-75af-437a-9c2e-cef28b4cbecc None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:41.750997 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62365e-75af-437a-9c2e-cef28b4cbecc None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:41.751281 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62365e-75af-437a-9c2e-cef28b4cbecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:41.773773 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-4d62365e-75af-437a-9c2e-cef28b4cbecc None None] MFA Rules not processed for user `29225fc9f22f4ec7b2979334760fc5ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:41.775056 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4d62365e-75af-437a-9c2e-cef28b4cbecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:41.782267 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1205/2421] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:48.463363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538277e3-52b2-484f-82b2-fe2268956015 service nova] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:48.546246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538277e3-52b2-484f-82b2-fe2268956015 service nova] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:48.558743 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a067d-0378-4654-9669-eef2708f81a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:48.559554 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a067d-0378-4654-9669-eef2708f81a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:48.559842 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a067d-0378-4654-9669-eef2708f81a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:48.560162 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a067d-0378-4654-9669-eef2708f81a3 admin admin] PATH_INFO: `/v3/users/80e71dc1fa894d98bc57195fbf4c415c` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:48.561439 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a067d-0378-4654-9669-eef2708f81a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80e71dc1fa894d98bc57195fbf4c415c)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:48.563773 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a067d-0378-4654-9669-eef2708f81a3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:48.870295 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-998a067d-0378-4654-9669-eef2708f81a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80e71dc1fa894d98bc57195fbf4c415c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:48.870994 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-998a067d-0378-4654-9669-eef2708f81a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80e71dc1fa894d98bc57195fbf4c415c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:48.883158 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-998a067d-0378-4654-9669-eef2708f81a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80e71dc1fa894d98bc57195fbf4c415c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:48.889934 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-998a067d-0378-4654-9669-eef2708f81a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80e71dc1fa894d98bc57195fbf4c415c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:48.890579 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-998a067d-0378-4654-9669-eef2708f81a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80e71dc1fa894d98bc57195fbf4c415c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:48.982185 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1217/2422] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:55:48 2022] DELETE /identity/v3/users/80e71dc1fa894d98bc57195fbf4c415c => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:48.987557 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e4f42-f499-4ea9-9c6a-a01682b0f739 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:48.988037 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e4f42-f499-4ea9-9c6a-a01682b0f739 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:48.988135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e4f42-f499-4ea9-9c6a-a01682b0f739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:49.008064 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-d94e4f42-f499-4ea9-9c6a-a01682b0f739 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:49.009287 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d94e4f42-f499-4ea9-9c6a-a01682b0f739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:49.069931 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1206/2423] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:49.079936 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a067d-0378-4654-9669-eef2708f81a3 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:49.081464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-998a067d-0378-4654-9669-eef2708f81a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:49.144721 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a067d-0378-4654-9669-eef2708f81a3 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:49.148412 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63af33c-a785-4c37-ac28-914387ed416d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:49.149029 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63af33c-a785-4c37-ac28-914387ed416d service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:49.149314 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63af33c-a785-4c37-ac28-914387ed416d service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:49.149590 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63af33c-a785-4c37-ac28-914387ed416d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:49.164840 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63af33c-a785-4c37-ac28-914387ed416d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:49.166287 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63af33c-a785-4c37-ac28-914387ed416d service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:49.176365 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1218/2424] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:55:49.620285 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e4f42-f499-4ea9-9c6a-a01682b0f739 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:49.621875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d94e4f42-f499-4ea9-9c6a-a01682b0f739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:49.635346 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e4f42-f499-4ea9-9c6a-a01682b0f739 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:49.644369 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c2f3e-0d32-44dd-b47b-c4aec600315f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:49.644694 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c2f3e-0d32-44dd-b47b-c4aec600315f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:49.644977 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c2f3e-0d32-44dd-b47b-c4aec600315f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:49.645250 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c2f3e-0d32-44dd-b47b-c4aec600315f admin admin] PATH_INFO: `/v3/projects/846d2a9d718e4b2ebf48996a639d4299` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:49.646461 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c2f3e-0d32-44dd-b47b-c4aec600315f admin admin] RBAC: Authorizing `identity:delete_project(project_id=846d2a9d718e4b2ebf48996a639d4299)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:49.648496 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c2f3e-0d32-44dd-b47b-c4aec600315f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:49.756529 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:49.772215 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-496c2f3e-0d32-44dd-b47b-c4aec600315f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '846d2a9d718e4b2ebf48996a639d4299', 'request_id': 'req-496c2f3e-0d32-44dd-b47b-c4aec600315f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:49.777717 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-496c2f3e-0d32-44dd-b47b-c4aec600315f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '846d2a9d718e4b2ebf48996a639d4299', 'request_id': 'req-496c2f3e-0d32-44dd-b47b-c4aec600315f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:49.908937 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-496c2f3e-0d32-44dd-b47b-c4aec600315f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '846d2a9d718e4b2ebf48996a639d4299', 'request_id': 'req-496c2f3e-0d32-44dd-b47b-c4aec600315f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:49.920657 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-496c2f3e-0d32-44dd-b47b-c4aec600315f admin admin] The token cache is being invalidate because project 846d2a9d718e4b2ebf48996a639d4299 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:55:49.920949 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-496c2f3e-0d32-44dd-b47b-c4aec600315f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:49.921589 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-496c2f3e-0d32-44dd-b47b-c4aec600315f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:55:49.923723 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1207/2425] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:55:49 2022] DELETE /identity/v3/projects/846d2a9d718e4b2ebf48996a639d4299 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:49.933619 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1768a-ebf0-462d-ba8e-fc0e769eb739 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:49.934105 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1768a-ebf0-462d-ba8e-fc0e769eb739 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:49.934246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1768a-ebf0-462d-ba8e-fc0e769eb739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:49.957108 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f6f1768a-ebf0-462d-ba8e-fc0e769eb739 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:49.958617 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f6f1768a-ebf0-462d-ba8e-fc0e769eb739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:49.971149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:50.018627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1219/2426] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:50.026703 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493dce7-d8dc-4aac-bb04-4c56bb7a2517 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:50.027181 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493dce7-d8dc-4aac-bb04-4c56bb7a2517 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:50.027377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493dce7-d8dc-4aac-bb04-4c56bb7a2517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:50.079010 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-7493dce7-d8dc-4aac-bb04-4c56bb7a2517 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:50.079010 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7493dce7-d8dc-4aac-bb04-4c56bb7a2517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:50.079010 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1208/2427] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:50.082054 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1768a-ebf0-462d-ba8e-fc0e769eb739 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:50.091836 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1768a-ebf0-462d-ba8e-fc0e769eb739 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:50.096368 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3cde3-0f5a-4a20-8287-8139c4b20782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:50.096985 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3cde3-0f5a-4a20-8287-8139c4b20782 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:50.097265 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3cde3-0f5a-4a20-8287-8139c4b20782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:50.097537 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3cde3-0f5a-4a20-8287-8139c4b20782 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:50.098219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3cde3-0f5a-4a20-8287-8139c4b20782 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:55:50.098523 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3cde3-0f5a-4a20-8287-8139c4b20782 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:50.100147 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3cde3-0f5a-4a20-8287-8139c4b20782 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:50.112285 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1220/2428] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:55:50.116581 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493dce7-d8dc-4aac-bb04-4c56bb7a2517 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:50.126012 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493dce7-d8dc-4aac-bb04-4c56bb7a2517 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:50.136347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b18b676-efe6-4cb8-9c0c-09a4a56a2b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:50.136347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18b676-efe6-4cb8-9c0c-09a4a56a2b2f admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:50.136347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18b676-efe6-4cb8-9c0c-09a4a56a2b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:50.136347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18b676-efe6-4cb8-9c0c-09a4a56a2b2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:50.136714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b18b676-efe6-4cb8-9c0c-09a4a56a2b2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:50.136714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b18b676-efe6-4cb8-9c0c-09a4a56a2b2f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:50.350763 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1209/2429] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:55:50 2022] POST /identity/v3/projects => generated 397 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:55:50.355209 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3cde3-0f5a-4a20-8287-8139c4b20782 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:50.366102 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3cde3-0f5a-4a20-8287-8139c4b20782 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:50.425297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd4254-d7f5-4220-a5bf-e532202ab7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:50.425994 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd4254-d7f5-4220-a5bf-e532202ab7ba admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:50.426285 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd4254-d7f5-4220-a5bf-e532202ab7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:50.426569 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd4254-d7f5-4220-a5bf-e532202ab7ba admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:50.427306 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd4254-d7f5-4220-a5bf-e532202ab7ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:50.429396 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd4254-d7f5-4220-a5bf-e532202ab7ba admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:50.543163 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1221/2430] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:55:50 2022] POST /identity/v3/users => generated 299 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:55:50.547559 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b18b676-efe6-4cb8-9c0c-09a4a56a2b2f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:50.558301 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b18b676-efe6-4cb8-9c0c-09a4a56a2b2f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:50.562929 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49161e-ab29-47ce-afc2-86aa5bcecb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:50.564178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49161e-ab29-47ce-afc2-86aa5bcecb51 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:50.564481 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49161e-ab29-47ce-afc2-86aa5bcecb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:50.565665 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49161e-ab29-47ce-afc2-86aa5bcecb51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:50.565665 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c49161e-ab29-47ce-afc2-86aa5bcecb51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:50.567080 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c49161e-ab29-47ce-afc2-86aa5bcecb51 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:50.579921 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1210/2431] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:55:50.583423 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd4254-d7f5-4220-a5bf-e532202ab7ba admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:50.597393 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd4254-d7f5-4220-a5bf-e532202ab7ba admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:50.602386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95394ade-8e29-41b3-ad5d-1b98fd625eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:50.602977 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95394ade-8e29-41b3-ad5d-1b98fd625eb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:50.603256 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95394ade-8e29-41b3-ad5d-1b98fd625eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:50.603561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95394ade-8e29-41b3-ad5d-1b98fd625eb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:50.604614 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95394ade-8e29-41b3-ad5d-1b98fd625eb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:50.606363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95394ade-8e29-41b3-ad5d-1b98fd625eb0 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:50.623452 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1222/2432] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:55:50.624788 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49161e-ab29-47ce-afc2-86aa5bcecb51 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:50.638753 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49161e-ab29-47ce-afc2-86aa5bcecb51 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:50.644517 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e2287-0a8a-4e87-b3dd-c6225816ce82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:50.645322 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e2287-0a8a-4e87-b3dd-c6225816ce82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:50.645672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e2287-0a8a-4e87-b3dd-c6225816ce82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:50.646149 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e2287-0a8a-4e87-b3dd-c6225816ce82 admin admin] PATH_INFO: `/v3/projects/3f8865a65a8343c1b5e7f3c41723050e/users/b562b35c471b4bff8199cef59d6ee125/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:50.661430 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518e2287-0a8a-4e87-b3dd-c6225816ce82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f8865a65a8343c1b5e7f3c41723050e, user_id=b562b35c471b4bff8199cef59d6ee125, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:50.664367 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518e2287-0a8a-4e87-b3dd-c6225816ce82 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:50.801015 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1211/2433] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:55:50 2022] PUT /identity/v3/projects/3f8865a65a8343c1b5e7f3c41723050e/users/b562b35c471b4bff8199cef59d6ee125/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:55:50.812861 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980949c-2886-4203-8c25-1201977decf4 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:50.813150 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980949c-2886-4203-8c25-1201977decf4 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:50.813431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980949c-2886-4203-8c25-1201977decf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:50.841928 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d980949c-2886-4203-8c25-1201977decf4 None None] MFA Rules not processed for user `b562b35c471b4bff8199cef59d6ee125`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:50.843150 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d980949c-2886-4203-8c25-1201977decf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:50.920572 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1223/2434] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:55:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:55:50.955162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e2287-0a8a-4e87-b3dd-c6225816ce82 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:50.956878 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-518e2287-0a8a-4e87-b3dd-c6225816ce82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:51.018115 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e2287-0a8a-4e87-b3dd-c6225816ce82 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:51.022107 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e80e2-86f2-485f-b442-5c0a7a96a5b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:51.022754 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e80e2-86f2-485f-b442-5c0a7a96a5b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:51.023095 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e80e2-86f2-485f-b442-5c0a7a96a5b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:51.023377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e80e2-86f2-485f-b442-5c0a7a96a5b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:51.032609 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e80e2-86f2-485f-b442-5c0a7a96a5b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:51.034053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e80e2-86f2-485f-b442-5c0a7a96a5b8 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:51.043367 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1212/2435] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:50 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:56:00.912700 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=91,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:01.039068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=67,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:04.098414 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980949c-2886-4203-8c25-1201977decf4 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:04.099971 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d980949c-2886-4203-8c25-1201977decf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:04.165711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980949c-2886-4203-8c25-1201977decf4 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:04.170328 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e34c87-098c-4b9f-9d34-c3474183274b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:04.170973 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e34c87-098c-4b9f-9d34-c3474183274b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:04.171258 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e34c87-098c-4b9f-9d34-c3474183274b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:04.171609 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e34c87-098c-4b9f-9d34-c3474183274b admin admin] PATH_INFO: `/v3/users/9afe7746763e448aaeb604741245850c` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:04.172783 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e34c87-098c-4b9f-9d34-c3474183274b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9afe7746763e448aaeb604741245850c)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:04.175255 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e34c87-098c-4b9f-9d34-c3474183274b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:04.513124 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-78e34c87-098c-4b9f-9d34-c3474183274b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9afe7746763e448aaeb604741245850c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:56:04.513124 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-78e34c87-098c-4b9f-9d34-c3474183274b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9afe7746763e448aaeb604741245850c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:56:04.513124 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-78e34c87-098c-4b9f-9d34-c3474183274b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9afe7746763e448aaeb604741245850c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:56:04.513124 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-78e34c87-098c-4b9f-9d34-c3474183274b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9afe7746763e448aaeb604741245850c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:56:04.513124 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-78e34c87-098c-4b9f-9d34-c3474183274b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9afe7746763e448aaeb604741245850c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:56:04.611826 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1224/2436] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:56:04 2022] DELETE /identity/v3/users/9afe7746763e448aaeb604741245850c => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:56:04.616416 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff54e71-5538-49bf-92eb-0c828cfe18c5 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:04.616938 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff54e71-5538-49bf-92eb-0c828cfe18c5 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:04.617129 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff54e71-5538-49bf-92eb-0c828cfe18c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:04.637080 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-1ff54e71-5538-49bf-92eb-0c828cfe18c5 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:56:04.638293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1ff54e71-5538-49bf-92eb-0c828cfe18c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:04.695951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1213/2437] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:56:04.706365 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e34c87-098c-4b9f-9d34-c3474183274b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:04.707799 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-78e34c87-098c-4b9f-9d34-c3474183274b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:04.764348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e34c87-098c-4b9f-9d34-c3474183274b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:04.768786 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792596a0-d8ec-464c-bdf6-3b48f70754a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:04.769399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792596a0-d8ec-464c-bdf6-3b48f70754a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:04.769682 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792596a0-d8ec-464c-bdf6-3b48f70754a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:04.769956 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792596a0-d8ec-464c-bdf6-3b48f70754a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:04.778630 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792596a0-d8ec-464c-bdf6-3b48f70754a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:04.780081 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792596a0-d8ec-464c-bdf6-3b48f70754a7 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:04.789439 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1225/2438] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:56:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:56:05.242303 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff54e71-5538-49bf-92eb-0c828cfe18c5 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:05.244101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1ff54e71-5538-49bf-92eb-0c828cfe18c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:05.261186 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff54e71-5538-49bf-92eb-0c828cfe18c5 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:05.266339 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f820eb6b-904d-4970-9391-7dad3028994a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:05.266689 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820eb6b-904d-4970-9391-7dad3028994a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:05.267691 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820eb6b-904d-4970-9391-7dad3028994a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:05.267875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820eb6b-904d-4970-9391-7dad3028994a admin admin] PATH_INFO: `/v3/projects/4705017825cc46139b39449c17f6c14f` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:05.269325 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f820eb6b-904d-4970-9391-7dad3028994a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4705017825cc46139b39449c17f6c14f)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:05.271239 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f820eb6b-904d-4970-9391-7dad3028994a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:05.384642 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f820eb6b-904d-4970-9391-7dad3028994a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4705017825cc46139b39449c17f6c14f', 'request_id': 'req-f820eb6b-904d-4970-9391-7dad3028994a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:56:05.390551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f820eb6b-904d-4970-9391-7dad3028994a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4705017825cc46139b39449c17f6c14f', 'request_id': 'req-f820eb6b-904d-4970-9391-7dad3028994a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:56:05.462550 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f820eb6b-904d-4970-9391-7dad3028994a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4705017825cc46139b39449c17f6c14f', 'request_id': 'req-f820eb6b-904d-4970-9391-7dad3028994a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:56:05.473932 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f820eb6b-904d-4970-9391-7dad3028994a admin admin] The token cache is being invalidate because project 4705017825cc46139b39449c17f6c14f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:56:05.474230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f820eb6b-904d-4970-9391-7dad3028994a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:56:05.474890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f820eb6b-904d-4970-9391-7dad3028994a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:56:05.476451 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1214/2439] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:56:05 2022] DELETE /identity/v3/projects/4705017825cc46139b39449c17f6c14f => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:56:05.481433 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b0896-f0d2-429d-9004-e7268c90765a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:05.482012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b0896-f0d2-429d-9004-e7268c90765a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:05.482012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b0896-f0d2-429d-9004-e7268c90765a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:05.502199 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-8c0b0896-f0d2-429d-9004-e7268c90765a None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:56:05.503504 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8c0b0896-f0d2-429d-9004-e7268c90765a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:05.522516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:05.568258 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1226/2440] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:56:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:56:05.576053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdbe01-3872-4c25-abc0-00ac69b3865b None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:05.576347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdbe01-3872-4c25-abc0-00ac69b3865b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:05.576630 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdbe01-3872-4c25-abc0-00ac69b3865b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:05.600173 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e6fdbe01-3872-4c25-abc0-00ac69b3865b None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:56:05.600173 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e6fdbe01-3872-4c25-abc0-00ac69b3865b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:05.607163 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1215/2441] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:56:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:56:05.611962 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b0896-f0d2-429d-9004-e7268c90765a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:05.621451 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b0896-f0d2-429d-9004-e7268c90765a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:05.626121 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2091e-53f3-4281-b7a8-c6147a14ab10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:05.632096 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2091e-53f3-4281-b7a8-c6147a14ab10 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:05.632096 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2091e-53f3-4281-b7a8-c6147a14ab10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:05.632096 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2091e-53f3-4281-b7a8-c6147a14ab10 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:05.632096 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2091e-53f3-4281-b7a8-c6147a14ab10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:56:05.632096 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2091e-53f3-4281-b7a8-c6147a14ab10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:05.632096 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2091e-53f3-4281-b7a8-c6147a14ab10 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:05.642472 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1227/2442] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 14:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:56:05.647992 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fdbe01-3872-4c25-abc0-00ac69b3865b None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:05.658320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fdbe01-3872-4c25-abc0-00ac69b3865b None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:05.664254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813cfe2-9582-48fc-8c1c-ba3c4c4f8e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:05.664887 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813cfe2-9582-48fc-8c1c-ba3c4c4f8e62 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:05.665172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813cfe2-9582-48fc-8c1c-ba3c4c4f8e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:05.670456 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813cfe2-9582-48fc-8c1c-ba3c4c4f8e62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:05.671155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4813cfe2-9582-48fc-8c1c-ba3c4c4f8e62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:05.673620 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4813cfe2-9582-48fc-8c1c-ba3c4c4f8e62 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:05.758993 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1216/2443] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 14:56:05 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:56:05.760477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2091e-53f3-4281-b7a8-c6147a14ab10 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:05.771665 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2091e-53f3-4281-b7a8-c6147a14ab10 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:05.838514 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7abfc-591f-46b7-b5b8-9ef040fa3cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:05.839132 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7abfc-591f-46b7-b5b8-9ef040fa3cf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:05.839441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7abfc-591f-46b7-b5b8-9ef040fa3cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:05.839721 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7abfc-591f-46b7-b5b8-9ef040fa3cf7 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:05.840381 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b7abfc-591f-46b7-b5b8-9ef040fa3cf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:05.842286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b7abfc-591f-46b7-b5b8-9ef040fa3cf7 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:05.926807 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1228/2444] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 14:56:05 2022] POST /identity/v3/users => generated 287 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:56:05.932783 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813cfe2-9582-48fc-8c1c-ba3c4c4f8e62 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:05.943712 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813cfe2-9582-48fc-8c1c-ba3c4c4f8e62 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:05.948381 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12401cb0-3511-4d19-82a9-7c7317f583bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:05.948991 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12401cb0-3511-4d19-82a9-7c7317f583bd admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:05.949273 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12401cb0-3511-4d19-82a9-7c7317f583bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:05.949551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12401cb0-3511-4d19-82a9-7c7317f583bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:05.950245 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12401cb0-3511-4d19-82a9-7c7317f583bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:05.951878 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12401cb0-3511-4d19-82a9-7c7317f583bd admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:05.962364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1217/2445] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:56:05.967010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7abfc-591f-46b7-b5b8-9ef040fa3cf7 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:05.977022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7abfc-591f-46b7-b5b8-9ef040fa3cf7 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:05.981725 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bc9e7-8fca-4441-ae5e-db7e9292c450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:05.982335 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bc9e7-8fca-4441-ae5e-db7e9292c450 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:05.982622 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bc9e7-8fca-4441-ae5e-db7e9292c450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:05.982904 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bc9e7-8fca-4441-ae5e-db7e9292c450 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:05.983631 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93bc9e7-8fca-4441-ae5e-db7e9292c450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:05.985231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93bc9e7-8fca-4441-ae5e-db7e9292c450 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:05.996153 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1229/2446] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 14:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:56:06.003356 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12401cb0-3511-4d19-82a9-7c7317f583bd admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:06.013551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12401cb0-3511-4d19-82a9-7c7317f583bd admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:06.019277 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38165d-fea7-43dc-b096-587414d6a9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:06.020133 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38165d-fea7-43dc-b096-587414d6a9a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:06.020479 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38165d-fea7-43dc-b096-587414d6a9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:06.020769 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38165d-fea7-43dc-b096-587414d6a9a7 admin admin] PATH_INFO: `/v3/projects/83e662c6c13145dea987e2c31ede8ee7/users/a296a71ae8c347338ce883ed1847ee5d/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:06.042201 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b38165d-fea7-43dc-b096-587414d6a9a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83e662c6c13145dea987e2c31ede8ee7, user_id=a296a71ae8c347338ce883ed1847ee5d, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:06.042402 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b38165d-fea7-43dc-b096-587414d6a9a7 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:06.165769 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1218/2447] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 14:56:06 2022] PUT /identity/v3/projects/83e662c6c13145dea987e2c31ede8ee7/users/a296a71ae8c347338ce883ed1847ee5d/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:56:06.176037 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd77bc3-48c4-4272-92d1-591c8cfb1a4b None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:06.176326 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd77bc3-48c4-4272-92d1-591c8cfb1a4b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:06.176609 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd77bc3-48c4-4272-92d1-591c8cfb1a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:06.204227 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-5cd77bc3-48c4-4272-92d1-591c8cfb1a4b None None] MFA Rules not processed for user `a296a71ae8c347338ce883ed1847ee5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:56:06.209962 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5cd77bc3-48c4-4272-92d1-591c8cfb1a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:06.269061 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1230/2448] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:56:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:56:06.280217 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38165d-fea7-43dc-b096-587414d6a9a7 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:06.281926 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1b38165d-fea7-43dc-b096-587414d6a9a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:06.304615 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:06.339193 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38165d-fea7-43dc-b096-587414d6a9a7 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:06.343554 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bd3910-1f79-4a30-9139-37785085f873 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:06.344174 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd3910-1f79-4a30-9139-37785085f873 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:06.344464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd3910-1f79-4a30-9139-37785085f873 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:06.344743 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd3910-1f79-4a30-9139-37785085f873 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:06.353122 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bd3910-1f79-4a30-9139-37785085f873 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:06.354559 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bd3910-1f79-4a30-9139-37785085f873 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:06.363853 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1219/2449] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:56:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:56:16.265260 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:16.428330 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=17 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:20.826443 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd77bc3-48c4-4272-92d1-591c8cfb1a4b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:20.827964 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5cd77bc3-48c4-4272-92d1-591c8cfb1a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:20.896334 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd77bc3-48c4-4272-92d1-591c8cfb1a4b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:20.901038 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555bcb45-05c4-4b9b-aa78-b5f75bbfee33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:20.901775 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555bcb45-05c4-4b9b-aa78-b5f75bbfee33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:20.902058 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555bcb45-05c4-4b9b-aa78-b5f75bbfee33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:20.902330 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555bcb45-05c4-4b9b-aa78-b5f75bbfee33 admin admin] PATH_INFO: `/v3/users/b562b35c471b4bff8199cef59d6ee125` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:20.903547 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555bcb45-05c4-4b9b-aa78-b5f75bbfee33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b562b35c471b4bff8199cef59d6ee125)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:20.905502 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555bcb45-05c4-4b9b-aa78-b5f75bbfee33 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:21.147032 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-555bcb45-05c4-4b9b-aa78-b5f75bbfee33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b562b35c471b4bff8199cef59d6ee125'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:56:21.147767 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-555bcb45-05c4-4b9b-aa78-b5f75bbfee33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b562b35c471b4bff8199cef59d6ee125'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:56:21.159796 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-555bcb45-05c4-4b9b-aa78-b5f75bbfee33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b562b35c471b4bff8199cef59d6ee125'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:56:21.166860 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-555bcb45-05c4-4b9b-aa78-b5f75bbfee33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b562b35c471b4bff8199cef59d6ee125'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:56:21.167722 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-555bcb45-05c4-4b9b-aa78-b5f75bbfee33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b562b35c471b4bff8199cef59d6ee125'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:56:21.240508 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1231/2450] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:56:20 2022] DELETE /identity/v3/users/b562b35c471b4bff8199cef59d6ee125 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:56:21.245245 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5046dcd-f90f-4636-9b48-e2297c188493 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:21.245759 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5046dcd-f90f-4636-9b48-e2297c188493 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:21.245971 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5046dcd-f90f-4636-9b48-e2297c188493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:21.266470 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-f5046dcd-f90f-4636-9b48-e2297c188493 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:56:21.268699 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f5046dcd-f90f-4636-9b48-e2297c188493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:21.332067 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1220/2451] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:56:21.342085 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555bcb45-05c4-4b9b-aa78-b5f75bbfee33 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:21.343791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-555bcb45-05c4-4b9b-aa78-b5f75bbfee33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:21.408358 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555bcb45-05c4-4b9b-aa78-b5f75bbfee33 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:21.412517 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675f233-b227-4be7-b7e5-4c58f472e0b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:21.413159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675f233-b227-4be7-b7e5-4c58f472e0b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:21.413445 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675f233-b227-4be7-b7e5-4c58f472e0b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:21.413723 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675f233-b227-4be7-b7e5-4c58f472e0b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:21.424383 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675f233-b227-4be7-b7e5-4c58f472e0b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:21.425854 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675f233-b227-4be7-b7e5-4c58f472e0b4 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:21.438111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1232/2452] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:56:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:56:22.208746 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5046dcd-f90f-4636-9b48-e2297c188493 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:22.210178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f5046dcd-f90f-4636-9b48-e2297c188493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:22.223858 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5046dcd-f90f-4636-9b48-e2297c188493 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:22.229154 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c507c-c02e-4a4e-9974-76c19a62465b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:22.230486 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c507c-c02e-4a4e-9974-76c19a62465b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:22.230486 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c507c-c02e-4a4e-9974-76c19a62465b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:22.230707 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c507c-c02e-4a4e-9974-76c19a62465b admin admin] PATH_INFO: `/v3/projects/3f8865a65a8343c1b5e7f3c41723050e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:22.234605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c507c-c02e-4a4e-9974-76c19a62465b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f8865a65a8343c1b5e7f3c41723050e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:22.234605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c507c-c02e-4a4e-9974-76c19a62465b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:22.405705 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-5b9c507c-c02e-4a4e-9974-76c19a62465b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f8865a65a8343c1b5e7f3c41723050e', 'request_id': 'req-5b9c507c-c02e-4a4e-9974-76c19a62465b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:56:22.412916 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-5b9c507c-c02e-4a4e-9974-76c19a62465b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f8865a65a8343c1b5e7f3c41723050e', 'request_id': 'req-5b9c507c-c02e-4a4e-9974-76c19a62465b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:56:22.520636 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-5b9c507c-c02e-4a4e-9974-76c19a62465b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f8865a65a8343c1b5e7f3c41723050e', 'request_id': 'req-5b9c507c-c02e-4a4e-9974-76c19a62465b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:56:22.543944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-5b9c507c-c02e-4a4e-9974-76c19a62465b admin admin] The token cache is being invalidate because project 3f8865a65a8343c1b5e7f3c41723050e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:56:22.544229 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-5b9c507c-c02e-4a4e-9974-76c19a62465b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:56:22.545894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-5b9c507c-c02e-4a4e-9974-76c19a62465b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:56:22.546629 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1221/2453] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:56:22 2022] DELETE /identity/v3/projects/3f8865a65a8343c1b5e7f3c41723050e => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:56:23.533910 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7773751-7d6a-4b75-b75f-a3b79616e4f0 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:23.534506 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7773751-7d6a-4b75-b75f-a3b79616e4f0 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:23.534506 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7773751-7d6a-4b75-b75f-a3b79616e4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:23.555060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d7773751-7d6a-4b75-b75f-a3b79616e4f0 None None] MFA Rules not processed for user `de3b593763e44e28be0f51187defa75d`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:56:23.559527 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d7773751-7d6a-4b75-b75f-a3b79616e4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:23.574481 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:23.616394 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1233/2454] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:56:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:56:32.540353 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:33.727000 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=15 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:45.915308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c507c-c02e-4a4e-9974-76c19a62465b admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:45.917468 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5b9c507c-c02e-4a4e-9974-76c19a62465b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:45.985381 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c507c-c02e-4a4e-9974-76c19a62465b admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:45.995441 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0251165-7ce0-4f9d-b19f-29404c88b5ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:45.996089 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0251165-7ce0-4f9d-b19f-29404c88b5ba service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:45.996377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0251165-7ce0-4f9d-b19f-29404c88b5ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:45.996659 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0251165-7ce0-4f9d-b19f-29404c88b5ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:45.998541 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a0251165-7ce0-4f9d-b19f-29404c88b5ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:46.085254 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0251165-7ce0-4f9d-b19f-29404c88b5ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:46.086731 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0251165-7ce0-4f9d-b19f-29404c88b5ba service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:46.101605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1222/2455] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:56:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:56:48.155590 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7773751-7d6a-4b75-b75f-a3b79616e4f0 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:48.157278 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d7773751-7d6a-4b75-b75f-a3b79616e4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:48.172067 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7773751-7d6a-4b75-b75f-a3b79616e4f0 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:48.177215 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed058c-efaa-43fb-a8f2-d77ab97c7623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:48.177847 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed058c-efaa-43fb-a8f2-d77ab97c7623 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:48.178130 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed058c-efaa-43fb-a8f2-d77ab97c7623 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:48.178473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed058c-efaa-43fb-a8f2-d77ab97c7623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:48.180350 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-27ed058c-efaa-43fb-a8f2-d77ab97c7623 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:48.194979 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed058c-efaa-43fb-a8f2-d77ab97c7623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:48.196559 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed058c-efaa-43fb-a8f2-d77ab97c7623 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:48.209359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1234/2456] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:56:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:56:51.497042 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0251165-7ce0-4f9d-b19f-29404c88b5ba service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:51.498740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a0251165-7ce0-4f9d-b19f-29404c88b5ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:51.569132 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0251165-7ce0-4f9d-b19f-29404c88b5ba service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:51.574491 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3ff8b-4066-4461-9704-d2f90b6ac309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:51.575135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3ff8b-4066-4461-9704-d2f90b6ac309 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:51.575457 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3ff8b-4066-4461-9704-d2f90b6ac309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:51.575755 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3ff8b-4066-4461-9704-d2f90b6ac309 admin admin] PATH_INFO: `/v3/users/a296a71ae8c347338ce883ed1847ee5d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:51.576928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb3ff8b-4066-4461-9704-d2f90b6ac309 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a296a71ae8c347338ce883ed1847ee5d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:51.578858 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb3ff8b-4066-4461-9704-d2f90b6ac309 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:51.819528 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:51.822368 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bfb3ff8b-4066-4461-9704-d2f90b6ac309 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a296a71ae8c347338ce883ed1847ee5d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:56:51.823156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bfb3ff8b-4066-4461-9704-d2f90b6ac309 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a296a71ae8c347338ce883ed1847ee5d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:56:51.836438 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bfb3ff8b-4066-4461-9704-d2f90b6ac309 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a296a71ae8c347338ce883ed1847ee5d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:56:51.844253 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bfb3ff8b-4066-4461-9704-d2f90b6ac309 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a296a71ae8c347338ce883ed1847ee5d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:56:51.845072 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bfb3ff8b-4066-4461-9704-d2f90b6ac309 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a296a71ae8c347338ce883ed1847ee5d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:56:51.945008 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1223/2457] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:56:51 2022] DELETE /identity/v3/users/a296a71ae8c347338ce883ed1847ee5d => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:56:51.949813 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08398a44-8362-4a38-86f9-11c6f2923838 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:51.950373 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08398a44-8362-4a38-86f9-11c6f2923838 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:51.950484 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08398a44-8362-4a38-86f9-11c6f2923838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:51.973225 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-08398a44-8362-4a38-86f9-11c6f2923838 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:56:51.974582 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-08398a44-8362-4a38-86f9-11c6f2923838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:52.044091 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1235/2458] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:56:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:56:52.055120 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3ff8b-4066-4461-9704-d2f90b6ac309 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:52.056756 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-bfb3ff8b-4066-4461-9704-d2f90b6ac309 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:52.119562 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3ff8b-4066-4461-9704-d2f90b6ac309 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:52.123976 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b4676-0f8a-4db5-8027-a5bea924203a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:52.124618 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b4676-0f8a-4db5-8027-a5bea924203a service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:52.124902 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b4676-0f8a-4db5-8027-a5bea924203a service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:52.125174 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b4676-0f8a-4db5-8027-a5bea924203a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:52.134937 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b4676-0f8a-4db5-8027-a5bea924203a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:52.136559 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b4676-0f8a-4db5-8027-a5bea924203a service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:52.148400 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1224/2459] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:56:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:56:53.038926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08398a44-8362-4a38-86f9-11c6f2923838 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:53.042644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-08398a44-8362-4a38-86f9-11c6f2923838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:53.068540 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08398a44-8362-4a38-86f9-11c6f2923838 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:53.074253 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f25b2f-e6fa-4224-b0b1-fd68db6ee6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:53.074947 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f25b2f-e6fa-4224-b0b1-fd68db6ee6a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:53.075232 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f25b2f-e6fa-4224-b0b1-fd68db6ee6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:53.075545 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f25b2f-e6fa-4224-b0b1-fd68db6ee6a0 admin admin] PATH_INFO: `/v3/projects/83e662c6c13145dea987e2c31ede8ee7` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:53.077043 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f25b2f-e6fa-4224-b0b1-fd68db6ee6a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83e662c6c13145dea987e2c31ede8ee7)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:53.082828 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f25b2f-e6fa-4224-b0b1-fd68db6ee6a0 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:53.295873 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-25f25b2f-e6fa-4224-b0b1-fd68db6ee6a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83e662c6c13145dea987e2c31ede8ee7', 'request_id': 'req-25f25b2f-e6fa-4224-b0b1-fd68db6ee6a0'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:56:53.301392 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-25f25b2f-e6fa-4224-b0b1-fd68db6ee6a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83e662c6c13145dea987e2c31ede8ee7', 'request_id': 'req-25f25b2f-e6fa-4224-b0b1-fd68db6ee6a0'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:56:53.429666 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-25f25b2f-e6fa-4224-b0b1-fd68db6ee6a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83e662c6c13145dea987e2c31ede8ee7', 'request_id': 'req-25f25b2f-e6fa-4224-b0b1-fd68db6ee6a0'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:56:53.443071 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-25f25b2f-e6fa-4224-b0b1-fd68db6ee6a0 admin admin] The token cache is being invalidate because project 83e662c6c13145dea987e2c31ede8ee7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:56:53.443370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-25f25b2f-e6fa-4224-b0b1-fd68db6ee6a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:56:53.444106 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-25f25b2f-e6fa-4224-b0b1-fd68db6ee6a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:56:53.445772 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1236/2460] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:56:53 2022] DELETE /identity/v3/projects/83e662c6c13145dea987e2c31ede8ee7 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:57:02.142548 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:DELETE=4,keystone:SELECT=39,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:03.439726 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:05.388481 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b4676-0f8a-4db5-8027-a5bea924203a service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:57:05.389939 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e05b4676-0f8a-4db5-8027-a5bea924203a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:05.403270 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b4676-0f8a-4db5-8027-a5bea924203a service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:57:05.466510 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e08b01-9dcf-482d-9048-e36c3b8dbf9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:57:05.467202 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e08b01-9dcf-482d-9048-e36c3b8dbf9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:05.467549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e08b01-9dcf-482d-9048-e36c3b8dbf9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:05.467809 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e08b01-9dcf-482d-9048-e36c3b8dbf9e admin admin] PATH_INFO: `/v3/users/06c447b856644b65aa2d07b44d10c871` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:05.468998 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e08b01-9dcf-482d-9048-e36c3b8dbf9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=06c447b856644b65aa2d07b44d10c871)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:57:05.471573 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e08b01-9dcf-482d-9048-e36c3b8dbf9e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:57:05.846163 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-30e08b01-9dcf-482d-9048-e36c3b8dbf9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06c447b856644b65aa2d07b44d10c871'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:57:05.847193 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-30e08b01-9dcf-482d-9048-e36c3b8dbf9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06c447b856644b65aa2d07b44d10c871'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:57:05.860006 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-30e08b01-9dcf-482d-9048-e36c3b8dbf9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06c447b856644b65aa2d07b44d10c871'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:57:05.868247 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-30e08b01-9dcf-482d-9048-e36c3b8dbf9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06c447b856644b65aa2d07b44d10c871'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:57:05.868954 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-30e08b01-9dcf-482d-9048-e36c3b8dbf9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06c447b856644b65aa2d07b44d10c871'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:57:06.029562 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1225/2461] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:57:05 2022] DELETE /identity/v3/users/06c447b856644b65aa2d07b44d10c871 => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:57:06.033999 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e661d84-c720-4414-a9a1-ad10edf84217 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:06.034819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e661d84-c720-4414-a9a1-ad10edf84217 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:06.035100 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e661d84-c720-4414-a9a1-ad10edf84217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:06.055163 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3e661d84-c720-4414-a9a1-ad10edf84217 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:57:06.056049 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3e661d84-c720-4414-a9a1-ad10edf84217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:06.114496 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1237/2462] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:57:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:57:06.124072 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e08b01-9dcf-482d-9048-e36c3b8dbf9e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:57:06.125459 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-30e08b01-9dcf-482d-9048-e36c3b8dbf9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:06.182153 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e08b01-9dcf-482d-9048-e36c3b8dbf9e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:57:06.185692 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8c595-4435-49f9-807b-56d4aedb6c58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:57:06.186304 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8c595-4435-49f9-807b-56d4aedb6c58 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:06.186586 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8c595-4435-49f9-807b-56d4aedb6c58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:06.186859 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8c595-4435-49f9-807b-56d4aedb6c58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:06.195915 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8c595-4435-49f9-807b-56d4aedb6c58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:57:06.197334 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8c595-4435-49f9-807b-56d4aedb6c58 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:57:06.207169 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1226/2463] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:57:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:57:06.866763 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e661d84-c720-4414-a9a1-ad10edf84217 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:57:06.868419 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3e661d84-c720-4414-a9a1-ad10edf84217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:06.882439 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e661d84-c720-4414-a9a1-ad10edf84217 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:57:06.888404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84242c57-37ca-4f68-8df3-5ec077c8f0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:57:06.889427 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84242c57-37ca-4f68-8df3-5ec077c8f0ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:06.889758 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84242c57-37ca-4f68-8df3-5ec077c8f0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:06.890108 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84242c57-37ca-4f68-8df3-5ec077c8f0ed admin admin] PATH_INFO: `/v3/projects/9b6cc17049d04b6c81762ebb834ca2e2` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:06.891253 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84242c57-37ca-4f68-8df3-5ec077c8f0ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b6cc17049d04b6c81762ebb834ca2e2)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:57:06.893379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84242c57-37ca-4f68-8df3-5ec077c8f0ed admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:57:07.065664 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-84242c57-37ca-4f68-8df3-5ec077c8f0ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b6cc17049d04b6c81762ebb834ca2e2', 'request_id': 'req-84242c57-37ca-4f68-8df3-5ec077c8f0ed'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:57:07.071128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-84242c57-37ca-4f68-8df3-5ec077c8f0ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b6cc17049d04b6c81762ebb834ca2e2', 'request_id': 'req-84242c57-37ca-4f68-8df3-5ec077c8f0ed'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:57:07.201457 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-84242c57-37ca-4f68-8df3-5ec077c8f0ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b6cc17049d04b6c81762ebb834ca2e2', 'request_id': 'req-84242c57-37ca-4f68-8df3-5ec077c8f0ed'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:57:07.214110 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-84242c57-37ca-4f68-8df3-5ec077c8f0ed admin admin] The token cache is being invalidate because project 9b6cc17049d04b6c81762ebb834ca2e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:57:07.214394 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-84242c57-37ca-4f68-8df3-5ec077c8f0ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:57:07.215030 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-84242c57-37ca-4f68-8df3-5ec077c8f0ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:57:07.216916 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1238/2464] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:57:06 2022] DELETE /identity/v3/projects/9b6cc17049d04b6c81762ebb834ca2e2 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:57:08.047338 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8c595-4435-49f9-807b-56d4aedb6c58 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:57:08.048847 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4eb8c595-4435-49f9-807b-56d4aedb6c58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:08.060240 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8c595-4435-49f9-807b-56d4aedb6c58 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:57:08.078141 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:08.109594 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e5096-db07-4f15-bc34-074ca52ef50b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:57:08.110216 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e5096-db07-4f15-bc34-074ca52ef50b service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:08.110528 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e5096-db07-4f15-bc34-074ca52ef50b service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:08.110808 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e5096-db07-4f15-bc34-074ca52ef50b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:08.112628 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-563e5096-db07-4f15-bc34-074ca52ef50b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:08.181484 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563e5096-db07-4f15-bc34-074ca52ef50b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:57:08.182975 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563e5096-db07-4f15-bc34-074ca52ef50b service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:57:08.194742 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1227/2465] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:57:08 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:57:17.210873 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:18.193490 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=40 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:42.448957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84242c57-37ca-4f68-8df3-5ec077c8f0ed admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:57:42.450362 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-84242c57-37ca-4f68-8df3-5ec077c8f0ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:42.523467 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84242c57-37ca-4f68-8df3-5ec077c8f0ed admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:57:42.528885 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f401bb-5731-4dfa-bc64-ebea8e78662b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:57:42.529499 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f401bb-5731-4dfa-bc64-ebea8e78662b service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:42.529785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f401bb-5731-4dfa-bc64-ebea8e78662b service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:42.530066 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f401bb-5731-4dfa-bc64-ebea8e78662b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:42.531868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d9f401bb-5731-4dfa-bc64-ebea8e78662b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:42.589710 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f401bb-5731-4dfa-bc64-ebea8e78662b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:57:42.591282 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f401bb-5731-4dfa-bc64-ebea8e78662b service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:57:42.600248 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1239/2466] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:57:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:57:48.721819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e5096-db07-4f15-bc34-074ca52ef50b service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:57:48.723016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-563e5096-db07-4f15-bc34-074ca52ef50b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:48.733861 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e5096-db07-4f15-bc34-074ca52ef50b service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:57:48.737541 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df895d57-9e55-4e0e-9dbe-533bf548a01a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:57:48.738157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df895d57-9e55-4e0e-9dbe-533bf548a01a service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:48.738433 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df895d57-9e55-4e0e-9dbe-533bf548a01a service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:48.738707 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df895d57-9e55-4e0e-9dbe-533bf548a01a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:48.740477 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-df895d57-9e55-4e0e-9dbe-533bf548a01a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:48.750377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df895d57-9e55-4e0e-9dbe-533bf548a01a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:57:48.752378 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df895d57-9e55-4e0e-9dbe-533bf548a01a service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:57:48.760944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1228/2467] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:57:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:57:52.596172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=56 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:58.757088 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=6 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:58:02.741640 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f401bb-5731-4dfa-bc64-ebea8e78662b service nova] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:58:02.743120 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d9f401bb-5731-4dfa-bc64-ebea8e78662b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:02.758566 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f401bb-5731-4dfa-bc64-ebea8e78662b service nova] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:58:02.764203 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1becd4d6-9e23-4a50-9564-bfe1bdbd3e29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:58:02.764814 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1becd4d6-9e23-4a50-9564-bfe1bdbd3e29 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:02.765105 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1becd4d6-9e23-4a50-9564-bfe1bdbd3e29 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:02.765384 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1becd4d6-9e23-4a50-9564-bfe1bdbd3e29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:02.767123 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1becd4d6-9e23-4a50-9564-bfe1bdbd3e29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:02.826329 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1becd4d6-9e23-4a50-9564-bfe1bdbd3e29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:58:02.827885 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1becd4d6-9e23-4a50-9564-bfe1bdbd3e29 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:58:02.839669 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1240/2468] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:58:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:58:12.834394 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=28 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:58:18.501249 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db61bde-65d7-451b-9980-4dd73efc376e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:18.501814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db61bde-65d7-451b-9980-4dd73efc376e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:18.501814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db61bde-65d7-451b-9980-4dd73efc376e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:18.522455 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-0db61bde-65d7-451b-9980-4dd73efc376e None None] MFA Rules not processed for user `19c0408b4321495eabd4738bbb8b711b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:58:18.523761 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0db61bde-65d7-451b-9980-4dd73efc376e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:18.531089 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1229/2469] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:58:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:58:22.138452 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1becd4d6-9e23-4a50-9564-bfe1bdbd3e29 service nova] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:58:22.153892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1becd4d6-9e23-4a50-9564-bfe1bdbd3e29 service nova] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:58:22.160829 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217a9d6-36f2-4628-987e-58ed3b577ee6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:58:22.161455 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217a9d6-36f2-4628-987e-58ed3b577ee6 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:22.161743 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217a9d6-36f2-4628-987e-58ed3b577ee6 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:22.162154 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217a9d6-36f2-4628-987e-58ed3b577ee6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:22.164665 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5217a9d6-36f2-4628-987e-58ed3b577ee6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:22.230888 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217a9d6-36f2-4628-987e-58ed3b577ee6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:58:22.232742 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217a9d6-36f2-4628-987e-58ed3b577ee6 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:58:22.242800 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1241/2470] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:58:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:58:26.841674 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db61bde-65d7-451b-9980-4dd73efc376e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:58:26.857079 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db61bde-65d7-451b-9980-4dd73efc376e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:58:26.864467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35985f-659f-4285-b41d-6d805f3ba96c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:58:26.865138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35985f-659f-4285-b41d-6d805f3ba96c service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:26.865429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35985f-659f-4285-b41d-6d805f3ba96c service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:26.865706 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35985f-659f-4285-b41d-6d805f3ba96c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:26.867657 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1e35985f-659f-4285-b41d-6d805f3ba96c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:26.888966 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35985f-659f-4285-b41d-6d805f3ba96c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:58:26.891530 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35985f-659f-4285-b41d-6d805f3ba96c service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:58:26.920018 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1230/2471] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:58:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:58:32.238092 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=28 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:58:36.901334 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=9 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:58:41.421033 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6dd985-d46c-47b3-8851-c3e3aadafcfb None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:41.421741 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6dd985-d46c-47b3-8851-c3e3aadafcfb None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:41.421741 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6dd985-d46c-47b3-8851-c3e3aadafcfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:41.441038 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3b6dd985-d46c-47b3-8851-c3e3aadafcfb None None] MFA Rules not processed for user `de3b593763e44e28be0f51187defa75d`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:58:41.442241 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3b6dd985-d46c-47b3-8851-c3e3aadafcfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:41.449272 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1242/2472] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:58:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:58:41.751113 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35985f-659f-4285-b41d-6d805f3ba96c service nova] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:58:41.760857 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35985f-659f-4285-b41d-6d805f3ba96c service nova] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:58:41.764611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761934f-339e-48f4-8a5a-c1389deed284 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:58:41.765237 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761934f-339e-48f4-8a5a-c1389deed284 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:41.766019 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761934f-339e-48f4-8a5a-c1389deed284 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:41.766320 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761934f-339e-48f4-8a5a-c1389deed284 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:41.768153 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f761934f-339e-48f4-8a5a-c1389deed284 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:41.778353 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f761934f-339e-48f4-8a5a-c1389deed284 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:58:41.782268 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f761934f-339e-48f4-8a5a-c1389deed284 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:58:41.788302 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1231/2473] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:58:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:58:42.204496 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6dd985-d46c-47b3-8851-c3e3aadafcfb None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:58:42.205880 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3b6dd985-d46c-47b3-8851-c3e3aadafcfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:42.279815 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6dd985-d46c-47b3-8851-c3e3aadafcfb None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:58:42.284906 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014f13f-c98f-4e97-9a55-d5954ad065f4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '252c2d2939134416a3e8efb254f03c5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:58:42.285526 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014f13f-c98f-4e97-9a55-d5954ad065f4 service placement] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:42.285812 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014f13f-c98f-4e97-9a55-d5954ad065f4 service placement] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:42.286094 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014f13f-c98f-4e97-9a55-d5954ad065f4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:42.287930 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a014f13f-c98f-4e97-9a55-d5954ad065f4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:42.299535 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a014f13f-c98f-4e97-9a55-d5954ad065f4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:58:42.301112 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a014f13f-c98f-4e97-9a55-d5954ad065f4 service placement] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:58:42.310902 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1243/2474] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:58:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:58:47.592704 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761934f-339e-48f4-8a5a-c1389deed284 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:58:47.606171 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761934f-339e-48f4-8a5a-c1389deed284 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:58:47.612034 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b936b-f353-4c3c-b6d3-4e9381e95d60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:58:47.612661 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b936b-f353-4c3c-b6d3-4e9381e95d60 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:47.612943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b936b-f353-4c3c-b6d3-4e9381e95d60 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:47.613217 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b936b-f353-4c3c-b6d3-4e9381e95d60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:47.614994 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e44b936b-f353-4c3c-b6d3-4e9381e95d60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:47.681333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b936b-f353-4c3c-b6d3-4e9381e95d60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:58:47.682872 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b936b-f353-4c3c-b6d3-4e9381e95d60 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:58:47.693125 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1232/2475] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:58:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:58:52.306751 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=35 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:58:56.551229 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014f13f-c98f-4e97-9a55-d5954ad065f4 service placement] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:58:56.565561 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014f13f-c98f-4e97-9a55-d5954ad065f4 service placement] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:58:56.571053 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe840c4-5afc-49a7-bf1e-3974c6c42cbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:58:56.571794 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe840c4-5afc-49a7-bf1e-3974c6c42cbf service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:56.572081 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe840c4-5afc-49a7-bf1e-3974c6c42cbf service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:56.572360 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe840c4-5afc-49a7-bf1e-3974c6c42cbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:56.574092 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6fe840c4-5afc-49a7-bf1e-3974c6c42cbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:56.588306 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe840c4-5afc-49a7-bf1e-3974c6c42cbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:58:56.589146 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe840c4-5afc-49a7-bf1e-3974c6c42cbf service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:58:56.600279 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1244/2476] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:58:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:58:57.408706 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b936b-f353-4c3c-b6d3-4e9381e95d60 service nova] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:58:57.417979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b936b-f353-4c3c-b6d3-4e9381e95d60 service nova] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:58:57.421587 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973af509-bce2-4349-b8fd-209ee11698a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:58:57.422193 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973af509-bce2-4349-b8fd-209ee11698a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:57.422479 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973af509-bce2-4349-b8fd-209ee11698a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:57.422752 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973af509-bce2-4349-b8fd-209ee11698a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:57.424525 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-973af509-bce2-4349-b8fd-209ee11698a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:57.487734 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973af509-bce2-4349-b8fd-209ee11698a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:58:57.489166 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973af509-bce2-4349-b8fd-209ee11698a0 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:58:57.498742 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1233/2477] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:58:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:59:01.446387 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe840c4-5afc-49a7-bf1e-3974c6c42cbf service nova] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:59:01.455482 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe840c4-5afc-49a7-bf1e-3974c6c42cbf service nova] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:59:01.461563 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a022a4-cdb4-4077-b36e-f6fc54ae43f5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '252c2d2939134416a3e8efb254f03c5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:59:01.461761 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a022a4-cdb4-4077-b36e-f6fc54ae43f5 service placement] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:59:01.462030 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a022a4-cdb4-4077-b36e-f6fc54ae43f5 service placement] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:59:01.462313 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a022a4-cdb4-4077-b36e-f6fc54ae43f5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:59:01.464254 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-24a022a4-cdb4-4077-b36e-f6fc54ae43f5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:59:01.480553 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a022a4-cdb4-4077-b36e-f6fc54ae43f5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:59:01.487447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a022a4-cdb4-4077-b36e-f6fc54ae43f5 service placement] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:59:01.495026 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1245/2478] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:59:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:59:01.622310 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973af509-bce2-4349-b8fd-209ee11698a0 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:59:01.636158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973af509-bce2-4349-b8fd-209ee11698a0 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:59:01.642900 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814458ff-9b73-41b3-8abf-3ce885f40512 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:59:01.643589 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814458ff-9b73-41b3-8abf-3ce885f40512 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:59:01.643879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814458ff-9b73-41b3-8abf-3ce885f40512 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:59:01.644160 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814458ff-9b73-41b3-8abf-3ce885f40512 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:59:01.646122 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-814458ff-9b73-41b3-8abf-3ce885f40512 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:59:01.660783 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814458ff-9b73-41b3-8abf-3ce885f40512 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:59:01.662408 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814458ff-9b73-41b3-8abf-3ce885f40512 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:59:01.674977 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1234/2479] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:59:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:59:05.425104 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a022a4-cdb4-4077-b36e-f6fc54ae43f5 service placement] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:59:05.437007 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a022a4-cdb4-4077-b36e-f6fc54ae43f5 service placement] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:59:05.442285 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95830cf3-de1b-4ce6-bc38-97d57e50e93d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '252c2d2939134416a3e8efb254f03c5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:59:05.443044 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95830cf3-de1b-4ce6-bc38-97d57e50e93d service placement] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:59:05.443334 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95830cf3-de1b-4ce6-bc38-97d57e50e93d service placement] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:59:05.443734 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95830cf3-de1b-4ce6-bc38-97d57e50e93d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:59:05.445515 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-95830cf3-de1b-4ce6-bc38-97d57e50e93d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:59:05.457495 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95830cf3-de1b-4ce6-bc38-97d57e50e93d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:59:05.459150 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95830cf3-de1b-4ce6-bc38-97d57e50e93d service placement] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:59:05.469950 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1246/2480] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:59:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:59:05.605322 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814458ff-9b73-41b3-8abf-3ce885f40512 service nova] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:59:05.606979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-814458ff-9b73-41b3-8abf-3ce885f40512 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:59:05.622705 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814458ff-9b73-41b3-8abf-3ce885f40512 service nova] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:59:05.627460 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ab21a-969d-4a2f-9465-bdab6f5fd879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:59:05.628997 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ab21a-969d-4a2f-9465-bdab6f5fd879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:59:05.628997 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ab21a-969d-4a2f-9465-bdab6f5fd879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:59:05.628997 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ab21a-969d-4a2f-9465-bdab6f5fd879 admin admin] PATH_INFO: `/v3/users/2ae70024479e488c90c6c1b955cb6471` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:59:05.630011 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ab21a-969d-4a2f-9465-bdab6f5fd879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ae70024479e488c90c6c1b955cb6471)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:59:05.632029 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ab21a-969d-4a2f-9465-bdab6f5fd879 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:59:05.913415 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3a4ab21a-969d-4a2f-9465-bdab6f5fd879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ae70024479e488c90c6c1b955cb6471'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:59:05.914087 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3a4ab21a-969d-4a2f-9465-bdab6f5fd879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ae70024479e488c90c6c1b955cb6471'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:59:05.924815 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:59:05.927033 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3a4ab21a-969d-4a2f-9465-bdab6f5fd879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ae70024479e488c90c6c1b955cb6471'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:59:05.933955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3a4ab21a-969d-4a2f-9465-bdab6f5fd879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ae70024479e488c90c6c1b955cb6471'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:59:05.934657 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3a4ab21a-969d-4a2f-9465-bdab6f5fd879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ae70024479e488c90c6c1b955cb6471'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:59:06.048230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1235/2481] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:59:05 2022] DELETE /identity/v3/users/2ae70024479e488c90c6c1b955cb6471 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:59:06.352335 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95830cf3-de1b-4ce6-bc38-97d57e50e93d service placement] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:59:06.366253 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95830cf3-de1b-4ce6-bc38-97d57e50e93d service placement] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:59:06.427089 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c95ac-42c0-4c66-954b-bbc174f1e2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:59:06.427764 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c95ac-42c0-4c66-954b-bbc174f1e2b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:59:06.427988 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c95ac-42c0-4c66-954b-bbc174f1e2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:59:06.428263 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c95ac-42c0-4c66-954b-bbc174f1e2b2 admin admin] PATH_INFO: `/v3/projects/e7107f9317c24cc7b972fdc6b8c01322` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:59:06.429415 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607c95ac-42c0-4c66-954b-bbc174f1e2b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7107f9317c24cc7b972fdc6b8c01322)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:59:06.431318 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607c95ac-42c0-4c66-954b-bbc174f1e2b2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:59:06.556763 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-607c95ac-42c0-4c66-954b-bbc174f1e2b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7107f9317c24cc7b972fdc6b8c01322', 'request_id': 'req-607c95ac-42c0-4c66-954b-bbc174f1e2b2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:59:06.563518 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-607c95ac-42c0-4c66-954b-bbc174f1e2b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7107f9317c24cc7b972fdc6b8c01322', 'request_id': 'req-607c95ac-42c0-4c66-954b-bbc174f1e2b2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:59:06.653215 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-607c95ac-42c0-4c66-954b-bbc174f1e2b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7107f9317c24cc7b972fdc6b8c01322', 'request_id': 'req-607c95ac-42c0-4c66-954b-bbc174f1e2b2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:59:06.665406 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-607c95ac-42c0-4c66-954b-bbc174f1e2b2 admin admin] The token cache is being invalidate because project e7107f9317c24cc7b972fdc6b8c01322 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:59:06.665723 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-607c95ac-42c0-4c66-954b-bbc174f1e2b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:59:06.666412 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-607c95ac-42c0-4c66-954b-bbc174f1e2b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:59:06.668009 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1247/2482] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:59:06 2022] DELETE /identity/v3/projects/e7107f9317c24cc7b972fdc6b8c01322 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:59:06.672412 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ab21a-969d-4a2f-9465-bdab6f5fd879 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:59:06.673811 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3a4ab21a-969d-4a2f-9465-bdab6f5fd879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:59:06.687518 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ab21a-969d-4a2f-9465-bdab6f5fd879 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:59:06.743068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faaff05-ad5b-431f-af73-11f3144f622f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:59:06.743790 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faaff05-ad5b-431f-af73-11f3144f622f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:59:06.744073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faaff05-ad5b-431f-af73-11f3144f622f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:59:06.744352 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faaff05-ad5b-431f-af73-11f3144f622f admin admin] PATH_INFO: `/v3/users/d46bafbe150a47de957347791d94ed45` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:59:06.745569 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faaff05-ad5b-431f-af73-11f3144f622f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d46bafbe150a47de957347791d94ed45)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:59:06.747548 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faaff05-ad5b-431f-af73-11f3144f622f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:59:06.912426 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2faaff05-ad5b-431f-af73-11f3144f622f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd46bafbe150a47de957347791d94ed45'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:59:06.913098 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2faaff05-ad5b-431f-af73-11f3144f622f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd46bafbe150a47de957347791d94ed45'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:59:06.925613 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2faaff05-ad5b-431f-af73-11f3144f622f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd46bafbe150a47de957347791d94ed45'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:59:06.932613 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2faaff05-ad5b-431f-af73-11f3144f622f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd46bafbe150a47de957347791d94ed45'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:59:06.933279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2faaff05-ad5b-431f-af73-11f3144f622f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd46bafbe150a47de957347791d94ed45'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:59:07.082661 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1236/2483] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 14:59:06 2022] DELETE /identity/v3/users/d46bafbe150a47de957347791d94ed45 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:59:07.347110 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c95ac-42c0-4c66-954b-bbc174f1e2b2 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:59:07.348602 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-607c95ac-42c0-4c66-954b-bbc174f1e2b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:59:07.363046 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c95ac-42c0-4c66-954b-bbc174f1e2b2 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:59:07.412577 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:59:07.422704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411475c4-1dc9-4c4b-b2f1-46b2a1a5ba69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:59:07.426961 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411475c4-1dc9-4c4b-b2f1-46b2a1a5ba69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:59:07.426961 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411475c4-1dc9-4c4b-b2f1-46b2a1a5ba69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:59:07.426961 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411475c4-1dc9-4c4b-b2f1-46b2a1a5ba69 admin admin] PATH_INFO: `/v3/projects/2bb78824c1344af8a6aea40818681920` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:59:07.426961 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411475c4-1dc9-4c4b-b2f1-46b2a1a5ba69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bb78824c1344af8a6aea40818681920)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:59:07.428094 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411475c4-1dc9-4c4b-b2f1-46b2a1a5ba69 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:59:07.550292 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-411475c4-1dc9-4c4b-b2f1-46b2a1a5ba69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bb78824c1344af8a6aea40818681920', 'request_id': 'req-411475c4-1dc9-4c4b-b2f1-46b2a1a5ba69'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:59:07.555677 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-411475c4-1dc9-4c4b-b2f1-46b2a1a5ba69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bb78824c1344af8a6aea40818681920', 'request_id': 'req-411475c4-1dc9-4c4b-b2f1-46b2a1a5ba69'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:59:07.635451 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-411475c4-1dc9-4c4b-b2f1-46b2a1a5ba69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bb78824c1344af8a6aea40818681920', 'request_id': 'req-411475c4-1dc9-4c4b-b2f1-46b2a1a5ba69'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:59:07.648933 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-411475c4-1dc9-4c4b-b2f1-46b2a1a5ba69 admin admin] The token cache is being invalidate because project 2bb78824c1344af8a6aea40818681920 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:59:07.649223 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-411475c4-1dc9-4c4b-b2f1-46b2a1a5ba69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:59:07.649885 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-411475c4-1dc9-4c4b-b2f1-46b2a1a5ba69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:59:07.651444 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1248/2484] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 14:59:07 2022] DELETE /identity/v3/projects/2bb78824c1344af8a6aea40818681920 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:59:17.078740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:DELETE=12,keystone:SELECT=60,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:59:17.645451 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:59:44.757200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39204a35-1154-4214-b0b0-3ab5a08cf53e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:59:44.758803 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39204a35-1154-4214-b0b0-3ab5a08cf53e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:59:44.758803 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39204a35-1154-4214-b0b0-3ab5a08cf53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:59:44.789783 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-39204a35-1154-4214-b0b0-3ab5a08cf53e None None] MFA Rules not processed for user `e0615d20057342cfbd16aa0ec94d1979`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:59:44.790933 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-39204a35-1154-4214-b0b0-3ab5a08cf53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:59:44.865132 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1237/2485] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 14:59:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:59:45.212790 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411475c4-1dc9-4c4b-b2f1-46b2a1a5ba69 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:59:45.214405 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-411475c4-1dc9-4c4b-b2f1-46b2a1a5ba69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:59:45.296519 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411475c4-1dc9-4c4b-b2f1-46b2a1a5ba69 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:59:45.301269 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2253b7-f8aa-4ccb-8f43-5f57922c757f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:59:45.301269 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2253b7-f8aa-4ccb-8f43-5f57922c757f service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:59:45.301499 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2253b7-f8aa-4ccb-8f43-5f57922c757f service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:59:45.301748 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2253b7-f8aa-4ccb-8f43-5f57922c757f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:59:45.303521 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8d2253b7-f8aa-4ccb-8f43-5f57922c757f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:59:45.314403 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2253b7-f8aa-4ccb-8f43-5f57922c757f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:59:45.315902 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2253b7-f8aa-4ccb-8f43-5f57922c757f service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:59:45.324886 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1249/2486] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:59:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:59:54.863015 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=37 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:59:55.322754 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=35 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:59:55.376707 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018863b-5643-4071-a0ac-0b66d04e4e28 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:59:55.377295 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018863b-5643-4071-a0ac-0b66d04e4e28 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:59:55.377853 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018863b-5643-4071-a0ac-0b66d04e4e28 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:59:55.380251 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1238/2487] 10.222.0.97 () {54 vars in 921 bytes} [Tue Jul 26 14:59:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:59:55.389867 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5a806-22c7-4fdc-b7a9-5ecd3f045aaa None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:59:55.391253 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5a806-22c7-4fdc-b7a9-5ecd3f045aaa None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:59:55.394442 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5a806-22c7-4fdc-b7a9-5ecd3f045aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:59:55.459032 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-73e5a806-22c7-4fdc-b7a9-5ecd3f045aaa None None] MFA Rules not processed for user `c81180fb4ff04c1186a21e43fb597a91`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:59:55.460299 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-73e5a806-22c7-4fdc-b7a9-5ecd3f045aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:59:55.467127 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1250/2488] 10.222.0.97 () {58 vars in 1018 bytes} [Tue Jul 26 14:59:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:59:55.479766 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018863b-5643-4071-a0ac-0b66d04e4e28 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:59:55.491172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018863b-5643-4071-a0ac-0b66d04e4e28 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:59:55.495306 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0866baf-a96b-4c8f-8ca0-178ab653fdf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:59:55.496022 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0866baf-a96b-4c8f-8ca0-178ab653fdf1 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:59:55.496314 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0866baf-a96b-4c8f-8ca0-178ab653fdf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:59:55.496590 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0866baf-a96b-4c8f-8ca0-178ab653fdf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:59:55.505814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0866baf-a96b-4c8f-8ca0-178ab653fdf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:59:55.507261 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0866baf-a96b-4c8f-8ca0-178ab653fdf1 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:59:55.517470 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1239/2489] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:59:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:59:55.602246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9be491-6575-49ce-aef9-351652769bbd None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:59:55.602534 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9be491-6575-49ce-aef9-351652769bbd None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:59:55.602808 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9be491-6575-49ce-aef9-351652769bbd None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:59:55.604232 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1251/2490] 10.222.0.97 () {54 vars in 921 bytes} [Tue Jul 26 14:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:59:55.609836 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9acabb-7041-468f-a0ad-352e22e7b145 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:59:55.610129 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9acabb-7041-468f-a0ad-352e22e7b145 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:59:55.617956 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9acabb-7041-468f-a0ad-352e22e7b145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:59:55.644536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-5c9acabb-7041-468f-a0ad-352e22e7b145 None None] MFA Rules not processed for user `266abef6cda5495b897170f92043aa03`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:59:55.645790 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5c9acabb-7041-468f-a0ad-352e22e7b145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:59:55.748196 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1240/2491] 10.222.0.97 () {58 vars in 1018 bytes} [Tue Jul 26 14:59:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:59:55.759040 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9be491-6575-49ce-aef9-351652769bbd None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:59:55.770641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9be491-6575-49ce-aef9-351652769bbd None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:59:55.774476 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f3615-e41a-4b78-bc9b-2d3c46e63551 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:59:55.775081 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f3615-e41a-4b78-bc9b-2d3c46e63551 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:59:55.775366 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f3615-e41a-4b78-bc9b-2d3c46e63551 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:59:55.775672 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f3615-e41a-4b78-bc9b-2d3c46e63551 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:59:55.786013 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387f3615-e41a-4b78-bc9b-2d3c46e63551 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:59:55.787511 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387f3615-e41a-4b78-bc9b-2d3c46e63551 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:59:55.798328 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1252/2492] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:59:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:59:59.028018 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9acabb-7041-468f-a0ad-352e22e7b145 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:59:59.050433 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9acabb-7041-468f-a0ad-352e22e7b145 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:59:59.054629 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fb4fe-ae3f-4cfb-bf8c-05ba1b7ab22c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:59:59.055368 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fb4fe-ae3f-4cfb-bf8c-05ba1b7ab22c service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:59:59.062746 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fb4fe-ae3f-4cfb-bf8c-05ba1b7ab22c service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:59:59.062746 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fb4fe-ae3f-4cfb-bf8c-05ba1b7ab22c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:59:59.062746 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-969fb4fe-ae3f-4cfb-bf8c-05ba1b7ab22c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:59:59.159326 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969fb4fe-ae3f-4cfb-bf8c-05ba1b7ab22c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:59:59.161745 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969fb4fe-ae3f-4cfb-bf8c-05ba1b7ab22c service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:59:59.183886 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1241/2493] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 14:59:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:00:03.792750 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f3615-e41a-4b78-bc9b-2d3c46e63551 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:00:03.794534 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-387f3615-e41a-4b78-bc9b-2d3c46e63551 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:00:03.811839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f3615-e41a-4b78-bc9b-2d3c46e63551 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:00:03.818885 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619ecae-fd54-467e-89c9-7cac29cba0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:00:03.819599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619ecae-fd54-467e-89c9-7cac29cba0b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:03.819875 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619ecae-fd54-467e-89c9-7cac29cba0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:03.820599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619ecae-fd54-467e-89c9-7cac29cba0b3 admin admin] PATH_INFO: `/v3/users/19c0408b4321495eabd4738bbb8b711b` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:03.822422 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c619ecae-fd54-467e-89c9-7cac29cba0b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19c0408b4321495eabd4738bbb8b711b)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:00:03.824648 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c619ecae-fd54-467e-89c9-7cac29cba0b3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:00:04.031496 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c619ecae-fd54-467e-89c9-7cac29cba0b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19c0408b4321495eabd4738bbb8b711b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:00:04.032469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c619ecae-fd54-467e-89c9-7cac29cba0b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19c0408b4321495eabd4738bbb8b711b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:00:04.053195 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c619ecae-fd54-467e-89c9-7cac29cba0b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19c0408b4321495eabd4738bbb8b711b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:00:04.061887 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c619ecae-fd54-467e-89c9-7cac29cba0b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19c0408b4321495eabd4738bbb8b711b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:00:04.062773 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c619ecae-fd54-467e-89c9-7cac29cba0b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19c0408b4321495eabd4738bbb8b711b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:00:04.201009 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1253/2494] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:00:03 2022] DELETE /identity/v3/users/19c0408b4321495eabd4738bbb8b711b => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:00:04.524164 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fb4fe-ae3f-4cfb-bf8c-05ba1b7ab22c service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:00:04.529726 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:00:04.536623 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fb4fe-ae3f-4cfb-bf8c-05ba1b7ab22c service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:00:04.601302 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28bbec5-23ac-4bf9-9144-fc5214b7a920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:00:04.601791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28bbec5-23ac-4bf9-9144-fc5214b7a920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:04.601953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28bbec5-23ac-4bf9-9144-fc5214b7a920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:04.602536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28bbec5-23ac-4bf9-9144-fc5214b7a920 admin admin] PATH_INFO: `/v3/projects/4f3f79533bbf48018798e6e65050d69c` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:04.604176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28bbec5-23ac-4bf9-9144-fc5214b7a920 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f3f79533bbf48018798e6e65050d69c)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:00:04.606153 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28bbec5-23ac-4bf9-9144-fc5214b7a920 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:00:04.775012 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d28bbec5-23ac-4bf9-9144-fc5214b7a920 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f3f79533bbf48018798e6e65050d69c', 'request_id': 'req-d28bbec5-23ac-4bf9-9144-fc5214b7a920'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:00:04.780672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d28bbec5-23ac-4bf9-9144-fc5214b7a920 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f3f79533bbf48018798e6e65050d69c', 'request_id': 'req-d28bbec5-23ac-4bf9-9144-fc5214b7a920'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:00:04.852622 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d28bbec5-23ac-4bf9-9144-fc5214b7a920 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f3f79533bbf48018798e6e65050d69c', 'request_id': 'req-d28bbec5-23ac-4bf9-9144-fc5214b7a920'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:00:04.865172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d28bbec5-23ac-4bf9-9144-fc5214b7a920 admin admin] The token cache is being invalidate because project 4f3f79533bbf48018798e6e65050d69c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:00:04.865463 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d28bbec5-23ac-4bf9-9144-fc5214b7a920 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:00:04.866122 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d28bbec5-23ac-4bf9-9144-fc5214b7a920 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:00:04.867683 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1242/2495] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:00:04 2022] DELETE /identity/v3/projects/4f3f79533bbf48018798e6e65050d69c => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:00:04.873649 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e737a4-78b6-4fe5-8aa9-f5ad655cac0e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:04.874131 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e737a4-78b6-4fe5-8aa9-f5ad655cac0e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:04.874215 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e737a4-78b6-4fe5-8aa9-f5ad655cac0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:04.925755 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-74e737a4-78b6-4fe5-8aa9-f5ad655cac0e None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:00:04.927436 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-74e737a4-78b6-4fe5-8aa9-f5ad655cac0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:00:04.996020 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1254/2496] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:00:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:00:05.004256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c6332-d99d-4fa7-8ed7-a2076f120153 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:05.004538 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c6332-d99d-4fa7-8ed7-a2076f120153 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:05.004823 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c6332-d99d-4fa7-8ed7-a2076f120153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:05.026153 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b98c6332-d99d-4fa7-8ed7-a2076f120153 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:00:05.027385 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b98c6332-d99d-4fa7-8ed7-a2076f120153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:00:05.035607 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1243/2497] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:00:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:00:05.040092 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e737a4-78b6-4fe5-8aa9-f5ad655cac0e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:00:05.050975 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e737a4-78b6-4fe5-8aa9-f5ad655cac0e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:00:05.056341 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db51febb-3a96-48f8-9ce7-44d20dcbba75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:00:05.057082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51febb-3a96-48f8-9ce7-44d20dcbba75 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:05.057466 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51febb-3a96-48f8-9ce7-44d20dcbba75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:05.057832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51febb-3a96-48f8-9ce7-44d20dcbba75 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:05.058623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db51febb-3a96-48f8-9ce7-44d20dcbba75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:00:05.058992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db51febb-3a96-48f8-9ce7-44d20dcbba75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:00:05.061590 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db51febb-3a96-48f8-9ce7-44d20dcbba75 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:00:05.071084 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:00:05.076265 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1255/2498] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 15:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:00:05.078840 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c6332-d99d-4fa7-8ed7-a2076f120153 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:00:05.095221 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c6332-d99d-4fa7-8ed7-a2076f120153 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:00:05.101274 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf45c22-5f5c-4899-a8be-2cba042b66a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:00:05.102022 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf45c22-5f5c-4899-a8be-2cba042b66a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:05.102961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf45c22-5f5c-4899-a8be-2cba042b66a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:05.102961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf45c22-5f5c-4899-a8be-2cba042b66a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:05.103720 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf45c22-5f5c-4899-a8be-2cba042b66a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:00:05.106534 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf45c22-5f5c-4899-a8be-2cba042b66a1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:00:05.210523 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1244/2499] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 15:00:05 2022] POST /identity/v3/projects => generated 389 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:00:05.214807 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db51febb-3a96-48f8-9ce7-44d20dcbba75 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:00:05.227293 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db51febb-3a96-48f8-9ce7-44d20dcbba75 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:00:05.307306 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea1fd4-2a85-4027-8d6d-4b404cdb59a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:00:05.311491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea1fd4-2a85-4027-8d6d-4b404cdb59a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:05.311491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea1fd4-2a85-4027-8d6d-4b404cdb59a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:05.311491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea1fd4-2a85-4027-8d6d-4b404cdb59a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:05.311491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea1fd4-2a85-4027-8d6d-4b404cdb59a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:00:05.312560 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea1fd4-2a85-4027-8d6d-4b404cdb59a1 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:00:05.414027 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1256/2500] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 15:00:05 2022] POST /identity/v3/users => generated 295 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:00:05.418270 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf45c22-5f5c-4899-a8be-2cba042b66a1 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:00:05.429194 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf45c22-5f5c-4899-a8be-2cba042b66a1 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:00:05.434634 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512872d-c7bb-4ab0-9741-461259874292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:00:05.435879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512872d-c7bb-4ab0-9741-461259874292 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:05.436165 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512872d-c7bb-4ab0-9741-461259874292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:05.436444 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512872d-c7bb-4ab0-9741-461259874292 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:05.437155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512872d-c7bb-4ab0-9741-461259874292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:00:05.438911 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512872d-c7bb-4ab0-9741-461259874292 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:00:05.453839 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1245/2501] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:00:05.455991 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea1fd4-2a85-4027-8d6d-4b404cdb59a1 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:00:05.467774 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea1fd4-2a85-4027-8d6d-4b404cdb59a1 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:00:05.472674 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342b4f2-b9de-4a05-8e65-5879b3fd3bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:00:05.473314 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342b4f2-b9de-4a05-8e65-5879b3fd3bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:05.473597 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342b4f2-b9de-4a05-8e65-5879b3fd3bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:05.473880 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342b4f2-b9de-4a05-8e65-5879b3fd3bd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:05.474581 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342b4f2-b9de-4a05-8e65-5879b3fd3bd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:00:05.476254 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342b4f2-b9de-4a05-8e65-5879b3fd3bd4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:00:05.491646 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1257/2502] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:00:05.492976 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512872d-c7bb-4ab0-9741-461259874292 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:00:05.504753 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512872d-c7bb-4ab0-9741-461259874292 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:00:05.509905 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be64cddb-a86d-418c-843d-e4a67b37d657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:00:05.510603 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64cddb-a86d-418c-843d-e4a67b37d657 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:05.510896 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64cddb-a86d-418c-843d-e4a67b37d657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:05.511154 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64cddb-a86d-418c-843d-e4a67b37d657 admin admin] PATH_INFO: `/v3/projects/7f82b8853b874e61820108bc16e8dc41/users/df675a52f9b245198897842f7a6ecd13/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:05.529811 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be64cddb-a86d-418c-843d-e4a67b37d657 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f82b8853b874e61820108bc16e8dc41, user_id=df675a52f9b245198897842f7a6ecd13, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:00:05.532730 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be64cddb-a86d-418c-843d-e4a67b37d657 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:00:05.665447 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1246/2503] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:00:05 2022] PUT /identity/v3/projects/7f82b8853b874e61820108bc16e8dc41/users/df675a52f9b245198897842f7a6ecd13/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:00:05.671166 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d53c4-9ee1-44e4-88c8-f36f307328d9 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:05.671477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d53c4-9ee1-44e4-88c8-f36f307328d9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:05.671757 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d53c4-9ee1-44e4-88c8-f36f307328d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:05.693034 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-e48d53c4-9ee1-44e4-88c8-f36f307328d9 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:00:05.694279 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e48d53c4-9ee1-44e4-88c8-f36f307328d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:00:05.765998 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1258/2504] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:00:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:00:05.777934 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be64cddb-a86d-418c-843d-e4a67b37d657 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:00:05.780007 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-be64cddb-a86d-418c-843d-e4a67b37d657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:00:05.818607 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:00:05.860333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be64cddb-a86d-418c-843d-e4a67b37d657 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:00:05.864749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016f83b-1b63-4828-8630-5f578bc45b33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:00:05.865379 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016f83b-1b63-4828-8630-5f578bc45b33 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:05.865660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016f83b-1b63-4828-8630-5f578bc45b33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:05.865935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016f83b-1b63-4828-8630-5f578bc45b33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:05.877093 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a016f83b-1b63-4828-8630-5f578bc45b33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:00:05.878603 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a016f83b-1b63-4828-8630-5f578bc45b33 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:00:05.890609 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1247/2505] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:00:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:00:12.394731 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1c0fb-efd4-44ae-87a7-e51b39f820d9 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:12.395254 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1c0fb-efd4-44ae-87a7-e51b39f820d9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:12.395552 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1c0fb-efd4-44ae-87a7-e51b39f820d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:12.434795 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-5ba1c0fb-efd4-44ae-87a7-e51b39f820d9 None None] MFA Rules not processed for user `df675a52f9b245198897842f7a6ecd13`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:00:12.436792 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5ba1c0fb-efd4-44ae-87a7-e51b39f820d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:00:12.531666 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1259/2506] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:00:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:00:12.550009 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016f83b-1b63-4828-8630-5f578bc45b33 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:00:12.552215 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a016f83b-1b63-4828-8630-5f578bc45b33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:00:12.735990 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016f83b-1b63-4828-8630-5f578bc45b33 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:00:12.752291 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97052b50-b510-46fe-9c21-c5e6f127212e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:00:12.753326 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97052b50-b510-46fe-9c21-c5e6f127212e service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:12.754333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97052b50-b510-46fe-9c21-c5e6f127212e service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:12.754333 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97052b50-b510-46fe-9c21-c5e6f127212e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:12.764495 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97052b50-b510-46fe-9c21-c5e6f127212e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:00:12.766303 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97052b50-b510-46fe-9c21-c5e6f127212e service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:00:12.778130 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1248/2507] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:00:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:00:16.073826 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba879ef-360e-4818-85f6-5cd96c6aaab3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:16.074955 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba879ef-360e-4818-85f6-5cd96c6aaab3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:16.075321 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba879ef-360e-4818-85f6-5cd96c6aaab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:16.101732 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-aba879ef-360e-4818-85f6-5cd96c6aaab3 None None] MFA Rules not processed for user `df675a52f9b245198897842f7a6ecd13`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:00:16.101732 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-aba879ef-360e-4818-85f6-5cd96c6aaab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:00:16.107634 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1260/2508] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:00:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:00:18.861692 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97052b50-b510-46fe-9c21-c5e6f127212e service nova] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:00:18.863273 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-97052b50-b510-46fe-9c21-c5e6f127212e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:00:18.956998 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97052b50-b510-46fe-9c21-c5e6f127212e service nova] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:00:18.963302 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49513f89-9bfd-47ee-b3c3-809067ed8cda service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '252c2d2939134416a3e8efb254f03c5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:00:18.964001 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49513f89-9bfd-47ee-b3c3-809067ed8cda service placement] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:18.964319 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49513f89-9bfd-47ee-b3c3-809067ed8cda service placement] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:18.964642 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49513f89-9bfd-47ee-b3c3-809067ed8cda service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:18.967033 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-49513f89-9bfd-47ee-b3c3-809067ed8cda service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:00:18.979839 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49513f89-9bfd-47ee-b3c3-809067ed8cda service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:00:18.982226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49513f89-9bfd-47ee-b3c3-809067ed8cda service placement] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:00:18.992695 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1249/2509] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:00:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:00:26.090042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=96,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:00:28.988442 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=93 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:00:52.949591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08549d93-8135-459c-9964-d55af17c579d None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:52.963628 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08549d93-8135-459c-9964-d55af17c579d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:52.963628 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08549d93-8135-459c-9964-d55af17c579d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:52.968738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-08549d93-8135-459c-9964-d55af17c579d None None] MFA Rules not processed for user `de3b593763e44e28be0f51187defa75d`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:00:52.969961 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-08549d93-8135-459c-9964-d55af17c579d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:00:53.024737 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1261/2510] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:00:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:01:03.017364 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=25 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:01:50.391450 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49513f89-9bfd-47ee-b3c3-809067ed8cda service placement] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:01:50.392957 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-49513f89-9bfd-47ee-b3c3-809067ed8cda service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:01:50.470031 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49513f89-9bfd-47ee-b3c3-809067ed8cda service placement] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:01:50.474280 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee64647-e3b9-4acb-a99f-0056f0275b79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:01:50.474891 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee64647-e3b9-4acb-a99f-0056f0275b79 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:01:50.475172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee64647-e3b9-4acb-a99f-0056f0275b79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:01:50.475475 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee64647-e3b9-4acb-a99f-0056f0275b79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:01:50.477227 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2ee64647-e3b9-4acb-a99f-0056f0275b79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:01:50.490417 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee64647-e3b9-4acb-a99f-0056f0275b79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:01:50.491879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee64647-e3b9-4acb-a99f-0056f0275b79 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:01:50.504221 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1250/2511] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:01:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:01:58.427073 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08549d93-8135-459c-9964-d55af17c579d None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:01:58.437606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08549d93-8135-459c-9964-d55af17c579d None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:01:58.441235 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452b045-2ed3-4b06-bf89-b3bf0ef75ebf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:01:58.441884 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452b045-2ed3-4b06-bf89-b3bf0ef75ebf service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:01:58.442168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452b045-2ed3-4b06-bf89-b3bf0ef75ebf service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:01:58.442483 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452b045-2ed3-4b06-bf89-b3bf0ef75ebf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:01:58.444272 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5452b045-2ed3-4b06-bf89-b3bf0ef75ebf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:01:58.504324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5452b045-2ed3-4b06-bf89-b3bf0ef75ebf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:01:58.506094 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5452b045-2ed3-4b06-bf89-b3bf0ef75ebf service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:01:58.515083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1262/2512] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:01:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:02:00.498910 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=35 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:02:00.737816 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee64647-e3b9-4acb-a99f-0056f0275b79 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:00.746449 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee64647-e3b9-4acb-a99f-0056f0275b79 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:00.750065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0505e-ec53-401c-a5ed-9ef11f764d78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:00.750671 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0505e-ec53-401c-a5ed-9ef11f764d78 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:00.750974 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0505e-ec53-401c-a5ed-9ef11f764d78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:00.751228 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0505e-ec53-401c-a5ed-9ef11f764d78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:00.753001 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-99d0505e-ec53-401c-a5ed-9ef11f764d78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:00.766188 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0505e-ec53-401c-a5ed-9ef11f764d78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:00.767631 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0505e-ec53-401c-a5ed-9ef11f764d78 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:00.779186 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1251/2513] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:02:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:02:06.355933 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452b045-2ed3-4b06-bf89-b3bf0ef75ebf service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:06.365803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452b045-2ed3-4b06-bf89-b3bf0ef75ebf service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:06.369737 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b7f35-b8d0-4eb3-bf87-227be8935126 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:06.370368 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b7f35-b8d0-4eb3-bf87-227be8935126 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:06.370803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b7f35-b8d0-4eb3-bf87-227be8935126 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:06.371088 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b7f35-b8d0-4eb3-bf87-227be8935126 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:06.372987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-368b7f35-b8d0-4eb3-bf87-227be8935126 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:06.386002 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b7f35-b8d0-4eb3-bf87-227be8935126 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:06.387530 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b7f35-b8d0-4eb3-bf87-227be8935126 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:06.398251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1263/2514] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:02:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:02:10.774381 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=6 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:02:14.963576 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0505e-ec53-401c-a5ed-9ef11f764d78 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:14.965098 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-99d0505e-ec53-401c-a5ed-9ef11f764d78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:14.989697 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0505e-ec53-401c-a5ed-9ef11f764d78 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:14.989697 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75d22b-01cd-4e9f-b6dc-ab2d1aafbe35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:14.989697 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75d22b-01cd-4e9f-b6dc-ab2d1aafbe35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:14.989697 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75d22b-01cd-4e9f-b6dc-ab2d1aafbe35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:14.990086 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75d22b-01cd-4e9f-b6dc-ab2d1aafbe35 admin admin] PATH_INFO: `/v3/users/29225fc9f22f4ec7b2979334760fc5ea` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:14.990086 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f75d22b-01cd-4e9f-b6dc-ab2d1aafbe35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29225fc9f22f4ec7b2979334760fc5ea)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:14.990433 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f75d22b-01cd-4e9f-b6dc-ab2d1aafbe35 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:15.246729 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0f75d22b-01cd-4e9f-b6dc-ab2d1aafbe35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29225fc9f22f4ec7b2979334760fc5ea'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:15.247786 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0f75d22b-01cd-4e9f-b6dc-ab2d1aafbe35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29225fc9f22f4ec7b2979334760fc5ea'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:15.263920 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0f75d22b-01cd-4e9f-b6dc-ab2d1aafbe35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29225fc9f22f4ec7b2979334760fc5ea'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:15.271718 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0f75d22b-01cd-4e9f-b6dc-ab2d1aafbe35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29225fc9f22f4ec7b2979334760fc5ea'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:15.275758 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0f75d22b-01cd-4e9f-b6dc-ab2d1aafbe35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29225fc9f22f4ec7b2979334760fc5ea'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:15.433561 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1252/2515] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:02:14 2022] DELETE /identity/v3/users/29225fc9f22f4ec7b2979334760fc5ea => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:02:15.689655 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b7f35-b8d0-4eb3-bf87-227be8935126 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:15.699307 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b7f35-b8d0-4eb3-bf87-227be8935126 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:15.765522 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc3014-69dd-4358-8449-e7b42eaa61ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:15.766091 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc3014-69dd-4358-8449-e7b42eaa61ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:15.766376 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc3014-69dd-4358-8449-e7b42eaa61ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:15.766657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc3014-69dd-4358-8449-e7b42eaa61ac admin admin] PATH_INFO: `/v3/projects/fde004be36ee4efeaa9ada32b4e2182c` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:15.767837 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddc3014-69dd-4358-8449-e7b42eaa61ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=fde004be36ee4efeaa9ada32b4e2182c)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:15.769761 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddc3014-69dd-4358-8449-e7b42eaa61ac admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:15.876927 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1ddc3014-69dd-4358-8449-e7b42eaa61ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fde004be36ee4efeaa9ada32b4e2182c', 'request_id': 'req-1ddc3014-69dd-4358-8449-e7b42eaa61ac'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:15.882262 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1ddc3014-69dd-4358-8449-e7b42eaa61ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fde004be36ee4efeaa9ada32b4e2182c', 'request_id': 'req-1ddc3014-69dd-4358-8449-e7b42eaa61ac'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:15.993278 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1ddc3014-69dd-4358-8449-e7b42eaa61ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fde004be36ee4efeaa9ada32b4e2182c', 'request_id': 'req-1ddc3014-69dd-4358-8449-e7b42eaa61ac'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:16.006314 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1ddc3014-69dd-4358-8449-e7b42eaa61ac admin admin] The token cache is being invalidate because project fde004be36ee4efeaa9ada32b4e2182c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:02:16.006602 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1ddc3014-69dd-4358-8449-e7b42eaa61ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:16.007302 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1ddc3014-69dd-4358-8449-e7b42eaa61ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:16.008927 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1264/2516] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:02:15 2022] DELETE /identity/v3/projects/fde004be36ee4efeaa9ada32b4e2182c => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:02:16.013551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75d22b-01cd-4e9f-b6dc-ab2d1aafbe35 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:16.014980 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0f75d22b-01cd-4e9f-b6dc-ab2d1aafbe35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:16.028488 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75d22b-01cd-4e9f-b6dc-ab2d1aafbe35 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:16.085703 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b83328-e9ca-434f-822a-3fbf88f4a265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:16.086373 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b83328-e9ca-434f-822a-3fbf88f4a265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:16.086687 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b83328-e9ca-434f-822a-3fbf88f4a265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:16.086965 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b83328-e9ca-434f-822a-3fbf88f4a265 admin admin] PATH_INFO: `/v3/users/96a877efdba748489e5a5bb3181d8d47` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:16.088177 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b83328-e9ca-434f-822a-3fbf88f4a265 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96a877efdba748489e5a5bb3181d8d47)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:16.092978 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b83328-e9ca-434f-822a-3fbf88f4a265 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:16.355221 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-28b83328-e9ca-434f-822a-3fbf88f4a265 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96a877efdba748489e5a5bb3181d8d47'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:16.355911 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-28b83328-e9ca-434f-822a-3fbf88f4a265 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96a877efdba748489e5a5bb3181d8d47'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:16.369534 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-28b83328-e9ca-434f-822a-3fbf88f4a265 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96a877efdba748489e5a5bb3181d8d47'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:16.386592 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-28b83328-e9ca-434f-822a-3fbf88f4a265 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96a877efdba748489e5a5bb3181d8d47'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:16.387331 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-28b83328-e9ca-434f-822a-3fbf88f4a265 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96a877efdba748489e5a5bb3181d8d47'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:16.389304 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:02:16.466171 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1253/2517] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:02:16 2022] DELETE /identity/v3/users/96a877efdba748489e5a5bb3181d8d47 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:02:16.735340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc3014-69dd-4358-8449-e7b42eaa61ac admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:16.736937 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1ddc3014-69dd-4358-8449-e7b42eaa61ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:16.751160 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc3014-69dd-4358-8449-e7b42eaa61ac admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:16.765523 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:02:16.815650 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b6ed5-a23d-4453-a81e-eea1b9226611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:16.816534 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b6ed5-a23d-4453-a81e-eea1b9226611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:16.816657 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b6ed5-a23d-4453-a81e-eea1b9226611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:16.816849 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b6ed5-a23d-4453-a81e-eea1b9226611 admin admin] PATH_INFO: `/v3/projects/3f7526cec950478aa97ec6dd5b4827f2` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:16.818070 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720b6ed5-a23d-4453-a81e-eea1b9226611 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f7526cec950478aa97ec6dd5b4827f2)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:16.820044 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720b6ed5-a23d-4453-a81e-eea1b9226611 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:16.948868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-720b6ed5-a23d-4453-a81e-eea1b9226611 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f7526cec950478aa97ec6dd5b4827f2', 'request_id': 'req-720b6ed5-a23d-4453-a81e-eea1b9226611'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:16.954544 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-720b6ed5-a23d-4453-a81e-eea1b9226611 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f7526cec950478aa97ec6dd5b4827f2', 'request_id': 'req-720b6ed5-a23d-4453-a81e-eea1b9226611'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:17.170007 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-720b6ed5-a23d-4453-a81e-eea1b9226611 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f7526cec950478aa97ec6dd5b4827f2', 'request_id': 'req-720b6ed5-a23d-4453-a81e-eea1b9226611'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:17.183709 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-720b6ed5-a23d-4453-a81e-eea1b9226611 admin admin] The token cache is being invalidate because project 3f7526cec950478aa97ec6dd5b4827f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:02:17.184020 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-720b6ed5-a23d-4453-a81e-eea1b9226611 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:17.184653 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-720b6ed5-a23d-4453-a81e-eea1b9226611 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:17.186143 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1265/2518] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:02:16 2022] DELETE /identity/v3/projects/3f7526cec950478aa97ec6dd5b4827f2 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:02:19.273450 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b83328-e9ca-434f-822a-3fbf88f4a265 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:19.275586 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-28b83328-e9ca-434f-822a-3fbf88f4a265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:19.332437 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b83328-e9ca-434f-822a-3fbf88f4a265 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:19.386435 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1dbd5-912f-4e41-be51-c61c89adc7a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:19.387067 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1dbd5-912f-4e41-be51-c61c89adc7a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:19.387352 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1dbd5-912f-4e41-be51-c61c89adc7a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:19.387774 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1dbd5-912f-4e41-be51-c61c89adc7a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:19.390413 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-64a1dbd5-912f-4e41-be51-c61c89adc7a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:19.454233 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a1dbd5-912f-4e41-be51-c61c89adc7a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:19.455732 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a1dbd5-912f-4e41-be51-c61c89adc7a4 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:19.465857 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1254/2519] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:02:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:02:23.532303 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b6ed5-a23d-4453-a81e-eea1b9226611 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:23.533750 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-720b6ed5-a23d-4453-a81e-eea1b9226611 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:23.548440 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b6ed5-a23d-4453-a81e-eea1b9226611 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:23.553149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ac1685-7123-403b-9b48-aa1097f391fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:23.553807 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac1685-7123-403b-9b48-aa1097f391fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:23.554091 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac1685-7123-403b-9b48-aa1097f391fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:23.554370 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac1685-7123-403b-9b48-aa1097f391fa admin admin] PATH_INFO: `/v3/users/e0615d20057342cfbd16aa0ec94d1979` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:23.695666 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ac1685-7123-403b-9b48-aa1097f391fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0615d20057342cfbd16aa0ec94d1979)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:23.697647 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ac1685-7123-403b-9b48-aa1097f391fa admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:23.909562 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-06ac1685-7123-403b-9b48-aa1097f391fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0615d20057342cfbd16aa0ec94d1979'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:23.910417 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-06ac1685-7123-403b-9b48-aa1097f391fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0615d20057342cfbd16aa0ec94d1979'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:23.923210 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-06ac1685-7123-403b-9b48-aa1097f391fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0615d20057342cfbd16aa0ec94d1979'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:23.931291 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-06ac1685-7123-403b-9b48-aa1097f391fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0615d20057342cfbd16aa0ec94d1979'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:23.932004 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-06ac1685-7123-403b-9b48-aa1097f391fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0615d20057342cfbd16aa0ec94d1979'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:24.062665 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1266/2520] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:02:23 2022] DELETE /identity/v3/users/e0615d20057342cfbd16aa0ec94d1979 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:02:24.394716 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1dbd5-912f-4e41-be51-c61c89adc7a4 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:24.406064 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1dbd5-912f-4e41-be51-c61c89adc7a4 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:24.479927 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:02:24.479927 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a682bf-5c81-4110-a25c-a6068fcc56fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:24.479927 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a682bf-5c81-4110-a25c-a6068fcc56fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:24.479927 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a682bf-5c81-4110-a25c-a6068fcc56fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:24.480459 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a682bf-5c81-4110-a25c-a6068fcc56fb admin admin] PATH_INFO: `/v3/projects/7ccd4ff518dc4c37bfb7f343ccc97206` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:24.488815 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a682bf-5c81-4110-a25c-a6068fcc56fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ccd4ff518dc4c37bfb7f343ccc97206)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:24.491213 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a682bf-5c81-4110-a25c-a6068fcc56fb admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:24.655913 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-24a682bf-5c81-4110-a25c-a6068fcc56fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ccd4ff518dc4c37bfb7f343ccc97206', 'request_id': 'req-24a682bf-5c81-4110-a25c-a6068fcc56fb'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:24.661424 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-24a682bf-5c81-4110-a25c-a6068fcc56fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ccd4ff518dc4c37bfb7f343ccc97206', 'request_id': 'req-24a682bf-5c81-4110-a25c-a6068fcc56fb'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:24.798088 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-24a682bf-5c81-4110-a25c-a6068fcc56fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ccd4ff518dc4c37bfb7f343ccc97206', 'request_id': 'req-24a682bf-5c81-4110-a25c-a6068fcc56fb'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:24.811041 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-24a682bf-5c81-4110-a25c-a6068fcc56fb admin admin] The token cache is being invalidate because project 7ccd4ff518dc4c37bfb7f343ccc97206 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:02:24.811336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-24a682bf-5c81-4110-a25c-a6068fcc56fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:24.812014 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-24a682bf-5c81-4110-a25c-a6068fcc56fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:24.813535 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1255/2521] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:02:24 2022] DELETE /identity/v3/projects/7ccd4ff518dc4c37bfb7f343ccc97206 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:02:24.818576 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ae37e-4c3a-4fb9-8a6f-8af1f875025d None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:24.819034 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ae37e-4c3a-4fb9-8a6f-8af1f875025d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:24.819153 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ae37e-4c3a-4fb9-8a6f-8af1f875025d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:24.833974 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:02:24.839609 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-eb6ae37e-4c3a-4fb9-8a6f-8af1f875025d None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:02:24.840878 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-eb6ae37e-4c3a-4fb9-8a6f-8af1f875025d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:24.901856 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1267/2522] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:02:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:02:24.910791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c9750-d509-4571-aabb-b13bfbde998f None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:24.911093 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c9750-d509-4571-aabb-b13bfbde998f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:24.911350 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c9750-d509-4571-aabb-b13bfbde998f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:24.930950 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-598c9750-d509-4571-aabb-b13bfbde998f None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:02:24.932400 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-598c9750-d509-4571-aabb-b13bfbde998f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:24.940410 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1256/2523] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:02:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:02:24.944577 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ae37e-4c3a-4fb9-8a6f-8af1f875025d None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:24.954738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ae37e-4c3a-4fb9-8a6f-8af1f875025d None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:24.959250 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d87ad0-5a50-4b8f-b6ff-251589c19f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:24.959893 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d87ad0-5a50-4b8f-b6ff-251589c19f7c admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:24.962516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d87ad0-5a50-4b8f-b6ff-251589c19f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:24.962516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d87ad0-5a50-4b8f-b6ff-251589c19f7c admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:24.962516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d87ad0-5a50-4b8f-b6ff-251589c19f7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:02:24.962516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d87ad0-5a50-4b8f-b6ff-251589c19f7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:24.963006 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d87ad0-5a50-4b8f-b6ff-251589c19f7c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:24.974391 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1268/2524] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 15:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:02:24.978665 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c9750-d509-4571-aabb-b13bfbde998f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:24.988085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c9750-d509-4571-aabb-b13bfbde998f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:24.994305 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f15e3-314f-4277-a370-a96bece40730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:24.994945 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f15e3-314f-4277-a370-a96bece40730 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:24.995228 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f15e3-314f-4277-a370-a96bece40730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:24.995537 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f15e3-314f-4277-a370-a96bece40730 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:24.996209 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f15e3-314f-4277-a370-a96bece40730 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:24.998168 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f15e3-314f-4277-a370-a96bece40730 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:25.117522 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1257/2525] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 15:02:24 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:02:25.118619 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d87ad0-5a50-4b8f-b6ff-251589c19f7c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:25.128783 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d87ad0-5a50-4b8f-b6ff-251589c19f7c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:25.189715 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d5773-6fdd-4051-8e66-79a505105742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:25.190341 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d5773-6fdd-4051-8e66-79a505105742 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:25.190629 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d5773-6fdd-4051-8e66-79a505105742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:25.191010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d5773-6fdd-4051-8e66-79a505105742 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:25.191720 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1d5773-6fdd-4051-8e66-79a505105742 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:25.193638 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1d5773-6fdd-4051-8e66-79a505105742 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:25.318772 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1269/2526] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 15:02:25 2022] POST /identity/v3/users => generated 287 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:02:25.323648 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f15e3-314f-4277-a370-a96bece40730 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:25.333853 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f15e3-314f-4277-a370-a96bece40730 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:25.339150 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c1d95c-f49c-43b7-97f8-72290e982497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:25.339821 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1d95c-f49c-43b7-97f8-72290e982497 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:25.340110 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1d95c-f49c-43b7-97f8-72290e982497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:25.340392 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1d95c-f49c-43b7-97f8-72290e982497 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:25.341116 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c1d95c-f49c-43b7-97f8-72290e982497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:25.342722 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c1d95c-f49c-43b7-97f8-72290e982497 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:25.353535 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1258/2527] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:02:25.357493 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d5773-6fdd-4051-8e66-79a505105742 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:25.367711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d5773-6fdd-4051-8e66-79a505105742 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:25.372507 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd24e4fb-e3ef-424e-b60d-18a364f6fea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:25.373120 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24e4fb-e3ef-424e-b60d-18a364f6fea2 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:25.373439 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24e4fb-e3ef-424e-b60d-18a364f6fea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:25.373719 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24e4fb-e3ef-424e-b60d-18a364f6fea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:25.374450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd24e4fb-e3ef-424e-b60d-18a364f6fea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:25.376148 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd24e4fb-e3ef-424e-b60d-18a364f6fea2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:25.386975 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1270/2528] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:02:25.391386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c1d95c-f49c-43b7-97f8-72290e982497 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:25.402366 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c1d95c-f49c-43b7-97f8-72290e982497 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:25.406921 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca295b27-03c9-4c89-a5e4-d943702474fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:25.407618 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca295b27-03c9-4c89-a5e4-d943702474fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:25.407905 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca295b27-03c9-4c89-a5e4-d943702474fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:25.408356 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca295b27-03c9-4c89-a5e4-d943702474fe admin admin] PATH_INFO: `/v3/projects/0c3dc09e6feb437f880edefe2a72d04e/users/34a9408ca7b44cd2a585544851faa4d9/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:25.423119 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca295b27-03c9-4c89-a5e4-d943702474fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c3dc09e6feb437f880edefe2a72d04e, user_id=34a9408ca7b44cd2a585544851faa4d9, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:25.425838 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca295b27-03c9-4c89-a5e4-d943702474fe admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:25.581849 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1259/2529] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:02:25 2022] PUT /identity/v3/projects/0c3dc09e6feb437f880edefe2a72d04e/users/34a9408ca7b44cd2a585544851faa4d9/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:02:25.593018 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d544d-4690-4a5d-9e0f-ea979ac3e462 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:25.598753 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d544d-4690-4a5d-9e0f-ea979ac3e462 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:25.598753 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d544d-4690-4a5d-9e0f-ea979ac3e462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:25.621554 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-988d544d-4690-4a5d-9e0f-ea979ac3e462 None None] MFA Rules not processed for user `34a9408ca7b44cd2a585544851faa4d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:02:25.622762 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-988d544d-4690-4a5d-9e0f-ea979ac3e462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:25.688996 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1271/2530] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:02:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:02:25.700954 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca295b27-03c9-4c89-a5e4-d943702474fe admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:25.705984 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ca295b27-03c9-4c89-a5e4-d943702474fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:25.777017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:02:25.781633 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca295b27-03c9-4c89-a5e4-d943702474fe admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:25.787250 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee6477-16f4-47d4-97a2-6c4823f41326 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:25.787896 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee6477-16f4-47d4-97a2-6c4823f41326 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:25.788188 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee6477-16f4-47d4-97a2-6c4823f41326 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:25.788464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee6477-16f4-47d4-97a2-6c4823f41326 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:25.796204 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee6477-16f4-47d4-97a2-6c4823f41326 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:25.797789 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee6477-16f4-47d4-97a2-6c4823f41326 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:25.806993 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1260/2531] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:02:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:02:26.031150 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d544d-4690-4a5d-9e0f-ea979ac3e462 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:26.035287 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:02:26.093516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d544d-4690-4a5d-9e0f-ea979ac3e462 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:26.098658 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397b646-8cf1-474e-980d-f3f546caf15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:26.099301 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397b646-8cf1-474e-980d-f3f546caf15c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:26.099615 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397b646-8cf1-474e-980d-f3f546caf15c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:26.099893 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397b646-8cf1-474e-980d-f3f546caf15c admin admin] PATH_INFO: `/v3/users/34a9408ca7b44cd2a585544851faa4d9` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:26.101060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397b646-8cf1-474e-980d-f3f546caf15c admin admin] RBAC: Authorizing `identity:delete_user(user_id=34a9408ca7b44cd2a585544851faa4d9)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:26.103521 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397b646-8cf1-474e-980d-f3f546caf15c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:26.420895 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3397b646-8cf1-474e-980d-f3f546caf15c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34a9408ca7b44cd2a585544851faa4d9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:26.421618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3397b646-8cf1-474e-980d-f3f546caf15c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34a9408ca7b44cd2a585544851faa4d9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:26.435668 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3397b646-8cf1-474e-980d-f3f546caf15c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34a9408ca7b44cd2a585544851faa4d9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:26.443871 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3397b646-8cf1-474e-980d-f3f546caf15c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34a9408ca7b44cd2a585544851faa4d9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:26.444615 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3397b646-8cf1-474e-980d-f3f546caf15c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34a9408ca7b44cd2a585544851faa4d9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:26.838634 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1272/2532] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:02:26 2022] DELETE /identity/v3/users/34a9408ca7b44cd2a585544851faa4d9 => generated 0 bytes in 808 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:02:26.843253 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb39c9a-1b40-412e-b046-51ee1f788663 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:26.843827 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb39c9a-1b40-412e-b046-51ee1f788663 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:26.843827 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb39c9a-1b40-412e-b046-51ee1f788663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:26.863548 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-cdb39c9a-1b40-412e-b046-51ee1f788663 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:02:26.864900 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-cdb39c9a-1b40-412e-b046-51ee1f788663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:26.930381 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1261/2533] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:02:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:02:26.940212 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397b646-8cf1-474e-980d-f3f546caf15c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:26.941597 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3397b646-8cf1-474e-980d-f3f546caf15c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:26.998607 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397b646-8cf1-474e-980d-f3f546caf15c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:27.002348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44629e43-f874-4e9d-874a-8ec4225aaf74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:27.002964 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44629e43-f874-4e9d-874a-8ec4225aaf74 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:27.003241 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44629e43-f874-4e9d-874a-8ec4225aaf74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:27.003543 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44629e43-f874-4e9d-874a-8ec4225aaf74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:27.013363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44629e43-f874-4e9d-874a-8ec4225aaf74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:27.014804 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44629e43-f874-4e9d-874a-8ec4225aaf74 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:27.025706 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1273/2534] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:02:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:02:27.548065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb39c9a-1b40-412e-b046-51ee1f788663 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:27.551518 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-cdb39c9a-1b40-412e-b046-51ee1f788663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:27.569072 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb39c9a-1b40-412e-b046-51ee1f788663 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:27.574437 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3514114-ba41-40f6-8fd5-073f1889f035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:27.575117 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3514114-ba41-40f6-8fd5-073f1889f035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:27.575462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3514114-ba41-40f6-8fd5-073f1889f035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:27.575660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3514114-ba41-40f6-8fd5-073f1889f035 admin admin] PATH_INFO: `/v3/projects/0c3dc09e6feb437f880edefe2a72d04e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:27.576912 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3514114-ba41-40f6-8fd5-073f1889f035 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c3dc09e6feb437f880edefe2a72d04e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:27.578778 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3514114-ba41-40f6-8fd5-073f1889f035 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:27.659120 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9ffb1-2944-454c-8c26-726dacaa6154 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:27.659610 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9ffb1-2944-454c-8c26-726dacaa6154 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:27.659704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9ffb1-2944-454c-8c26-726dacaa6154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:27.697948 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:02:27.713822 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-43c9ffb1-2944-454c-8c26-726dacaa6154 None None] MFA Rules not processed for user `5179c3c4eaaa41f3beb79ba7f1993478`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:02:27.715050 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-43c9ffb1-2944-454c-8c26-726dacaa6154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:27.736674 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e3514114-ba41-40f6-8fd5-073f1889f035 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c3dc09e6feb437f880edefe2a72d04e', 'request_id': 'req-e3514114-ba41-40f6-8fd5-073f1889f035'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:27.742649 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e3514114-ba41-40f6-8fd5-073f1889f035 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c3dc09e6feb437f880edefe2a72d04e', 'request_id': 'req-e3514114-ba41-40f6-8fd5-073f1889f035'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:27.781186 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1274/2535] 10.222.0.97 () {58 vars in 1009 bytes} [Tue Jul 26 15:02:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:02:27.833015 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e3514114-ba41-40f6-8fd5-073f1889f035 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c3dc09e6feb437f880edefe2a72d04e', 'request_id': 'req-e3514114-ba41-40f6-8fd5-073f1889f035'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:27.846256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e3514114-ba41-40f6-8fd5-073f1889f035 admin admin] The token cache is being invalidate because project 0c3dc09e6feb437f880edefe2a72d04e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:02:27.846879 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e3514114-ba41-40f6-8fd5-073f1889f035 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:27.847173 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e3514114-ba41-40f6-8fd5-073f1889f035 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:27.848714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1262/2536] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:02:27 2022] DELETE /identity/v3/projects/0c3dc09e6feb437f880edefe2a72d04e => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:02:27.855394 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f20cd-47d6-43fd-93e6-1c9a73a2b28b None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:27.855714 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f20cd-47d6-43fd-93e6-1c9a73a2b28b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:27.855992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f20cd-47d6-43fd-93e6-1c9a73a2b28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:27.875614 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d96f20cd-47d6-43fd-93e6-1c9a73a2b28b None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:02:27.876896 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d96f20cd-47d6-43fd-93e6-1c9a73a2b28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:27.937086 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1275/2537] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:02:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:02:27.945304 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f0949-1d2c-4a98-99c3-5590e57192ea None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:27.945593 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f0949-1d2c-4a98-99c3-5590e57192ea None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:27.945868 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f0949-1d2c-4a98-99c3-5590e57192ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:27.965847 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-bb4f0949-1d2c-4a98-99c3-5590e57192ea None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:02:27.967111 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-bb4f0949-1d2c-4a98-99c3-5590e57192ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:27.975066 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1263/2538] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:02:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:02:27.983342 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f20cd-47d6-43fd-93e6-1c9a73a2b28b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:27.993039 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f20cd-47d6-43fd-93e6-1c9a73a2b28b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:27.997700 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94c700-f7ee-4cfe-a8f4-91fb2a9bf6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:27.998304 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94c700-f7ee-4cfe-a8f4-91fb2a9bf6ca admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:27.998590 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94c700-f7ee-4cfe-a8f4-91fb2a9bf6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:27.998869 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94c700-f7ee-4cfe-a8f4-91fb2a9bf6ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:27.999647 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de94c700-f7ee-4cfe-a8f4-91fb2a9bf6ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:02:27.999954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de94c700-f7ee-4cfe-a8f4-91fb2a9bf6ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:28.001534 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de94c700-f7ee-4cfe-a8f4-91fb2a9bf6ca admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:28.013139 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1276/2539] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 15:02:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:02:28.027418 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f0949-1d2c-4a98-99c3-5590e57192ea None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:28.036859 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f0949-1d2c-4a98-99c3-5590e57192ea None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:28.042486 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d353a0-4326-4e09-b722-6231ccbb202e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:28.043046 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d353a0-4326-4e09-b722-6231ccbb202e admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:28.043330 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d353a0-4326-4e09-b722-6231ccbb202e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:28.043638 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d353a0-4326-4e09-b722-6231ccbb202e admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:28.044319 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d353a0-4326-4e09-b722-6231ccbb202e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:28.046224 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d353a0-4326-4e09-b722-6231ccbb202e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:28.069543 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94c700-f7ee-4cfe-a8f4-91fb2a9bf6ca admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:28.072136 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-de94c700-f7ee-4cfe-a8f4-91fb2a9bf6ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:28.165137 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94c700-f7ee-4cfe-a8f4-91fb2a9bf6ca admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:28.170257 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb0cbf-9eeb-4e83-901d-9a6bac9d54cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85122c4052fd4ba19a2dae185a707caf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:28.170902 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb0cbf-9eeb-4e83-901d-9a6bac9d54cc service swift] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:28.171254 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb0cbf-9eeb-4e83-901d-9a6bac9d54cc service swift] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:28.171591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb0cbf-9eeb-4e83-901d-9a6bac9d54cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:28.173371 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-34eb0cbf-9eeb-4e83-901d-9a6bac9d54cc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:28.178159 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:02:28.225947 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1264/2540] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 15:02:28 2022] POST /identity/v3/projects => generated 407 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:02:28.226974 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d353a0-4326-4e09-b722-6231ccbb202e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:28.231213 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb0cbf-9eeb-4e83-901d-9a6bac9d54cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:28.232833 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb0cbf-9eeb-4e83-901d-9a6bac9d54cc service swift] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:28.242241 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1277/2541] 10.222.0.97 () {60 vars in 1384 bytes} [Tue Jul 26 15:02:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 15:02:28.283874 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:02:28.298188 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d353a0-4326-4e09-b722-6231ccbb202e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:28.304991 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71e8bb-a3ca-4351-ab9f-d60d650ef760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:28.305866 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71e8bb-a3ca-4351-ab9f-d60d650ef760 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:28.306684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71e8bb-a3ca-4351-ab9f-d60d650ef760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:28.306684 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71e8bb-a3ca-4351-ab9f-d60d650ef760 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:28.306920 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc71e8bb-a3ca-4351-ab9f-d60d650ef760 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:28.308860 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc71e8bb-a3ca-4351-ab9f-d60d650ef760 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:28.461037 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1265/2542] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 15:02:28 2022] POST /identity/v3/users => generated 304 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:02:28.465682 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb0cbf-9eeb-4e83-901d-9a6bac9d54cc service swift] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:28.476912 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb0cbf-9eeb-4e83-901d-9a6bac9d54cc service swift] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:28.484111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5bd805-9c5a-46b8-80ca-219192fdcb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:28.484829 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5bd805-9c5a-46b8-80ca-219192fdcb41 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:28.485227 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5bd805-9c5a-46b8-80ca-219192fdcb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:28.485575 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5bd805-9c5a-46b8-80ca-219192fdcb41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:28.486283 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5bd805-9c5a-46b8-80ca-219192fdcb41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:28.488289 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5bd805-9c5a-46b8-80ca-219192fdcb41 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:28.504414 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71e8bb-a3ca-4351-ab9f-d60d650ef760 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:28.509619 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1278/2543] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:02:28.514967 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71e8bb-a3ca-4351-ab9f-d60d650ef760 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:28.524750 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc95374-63f7-4543-a081-d07a3c6da983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:28.525388 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc95374-63f7-4543-a081-d07a3c6da983 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:28.525669 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc95374-63f7-4543-a081-d07a3c6da983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:28.525978 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc95374-63f7-4543-a081-d07a3c6da983 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:28.526679 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc95374-63f7-4543-a081-d07a3c6da983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:28.528318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc95374-63f7-4543-a081-d07a3c6da983 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:28.544015 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1266/2544] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:02:28.544534 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5bd805-9c5a-46b8-80ca-219192fdcb41 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:28.558208 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5bd805-9c5a-46b8-80ca-219192fdcb41 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:28.563745 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb373a-2557-434f-97d2-0e67433152f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:28.564533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb373a-2557-434f-97d2-0e67433152f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:28.564899 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb373a-2557-434f-97d2-0e67433152f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:28.565187 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb373a-2557-434f-97d2-0e67433152f8 admin admin] PATH_INFO: `/v3/projects/fe5199415c124a4dae2e9f965e5f715f/users/2a74bb723b064c1b82c85eb878fab45f/roles/e033961a9363456a8510c34414dd839e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:28.582892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb373a-2557-434f-97d2-0e67433152f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe5199415c124a4dae2e9f965e5f715f, user_id=2a74bb723b064c1b82c85eb878fab45f, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:28.586293 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb373a-2557-434f-97d2-0e67433152f8 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:28.703207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1279/2545] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:02:28 2022] PUT /identity/v3/projects/fe5199415c124a4dae2e9f965e5f715f/users/2a74bb723b064c1b82c85eb878fab45f/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:02:28.715727 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022e67b-0f5c-4468-bccc-687fdb7cc42e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:28.715727 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022e67b-0f5c-4468-bccc-687fdb7cc42e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:28.715727 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022e67b-0f5c-4468-bccc-687fdb7cc42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:28.779356 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6022e67b-0f5c-4468-bccc-687fdb7cc42e None None] MFA Rules not processed for user `2a74bb723b064c1b82c85eb878fab45f`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:02:28.780663 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6022e67b-0f5c-4468-bccc-687fdb7cc42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:28.873510 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1267/2546] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:02:28 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:02:28.877783 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb373a-2557-434f-97d2-0e67433152f8 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:28.966119 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb373a-2557-434f-97d2-0e67433152f8 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:28.974224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775f274-f8c8-45b6-9d38-894b684db626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:28.974224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775f274-f8c8-45b6-9d38-894b684db626 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:28.974224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775f274-f8c8-45b6-9d38-894b684db626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:28.974224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775f274-f8c8-45b6-9d38-894b684db626 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:28.974701 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6775f274-f8c8-45b6-9d38-894b684db626 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:28.975769 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6775f274-f8c8-45b6-9d38-894b684db626 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:29.103698 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1280/2547] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 15:02:28 2022] POST /identity/v3/projects => generated 409 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:02:29.109517 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022e67b-0f5c-4468-bccc-687fdb7cc42e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:29.121943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022e67b-0f5c-4468-bccc-687fdb7cc42e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:29.188903 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5abc3-49af-40f1-aeb7-90f16dbde323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:29.189668 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5abc3-49af-40f1-aeb7-90f16dbde323 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:29.190048 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5abc3-49af-40f1-aeb7-90f16dbde323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:29.190398 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5abc3-49af-40f1-aeb7-90f16dbde323 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:29.191202 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5abc3-49af-40f1-aeb7-90f16dbde323 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:29.193420 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5abc3-49af-40f1-aeb7-90f16dbde323 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:29.317620 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1268/2548] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 15:02:29 2022] POST /identity/v3/users => generated 311 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:02:29.318992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775f274-f8c8-45b6-9d38-894b684db626 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:29.334142 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775f274-f8c8-45b6-9d38-894b684db626 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:29.341448 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b424e-316c-4048-ac9d-a2422adf958b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:29.342335 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b424e-316c-4048-ac9d-a2422adf958b admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:29.342742 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b424e-316c-4048-ac9d-a2422adf958b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:29.343165 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b424e-316c-4048-ac9d-a2422adf958b admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:29.344242 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8b424e-316c-4048-ac9d-a2422adf958b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:29.346734 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8b424e-316c-4048-ac9d-a2422adf958b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:29.362927 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1281/2549] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:02:29.373751 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5abc3-49af-40f1-aeb7-90f16dbde323 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:29.387389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5abc3-49af-40f1-aeb7-90f16dbde323 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:29.393401 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a06ba-0fff-41dd-b950-571149bdd3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:29.395191 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a06ba-0fff-41dd-b950-571149bdd3b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:29.395882 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a06ba-0fff-41dd-b950-571149bdd3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:29.396269 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a06ba-0fff-41dd-b950-571149bdd3b6 admin admin] PATH_INFO: `/v3/projects/6bed6a59cea946229b29e16bbe0900d8/users/3da20bf7a132489d994e84b4edbb5b6f/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:29.418634 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a06ba-0fff-41dd-b950-571149bdd3b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bed6a59cea946229b29e16bbe0900d8, user_id=3da20bf7a132489d994e84b4edbb5b6f, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:29.421694 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a06ba-0fff-41dd-b950-571149bdd3b6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:29.561031 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1269/2550] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:02:29 2022] PUT /identity/v3/projects/6bed6a59cea946229b29e16bbe0900d8/users/3da20bf7a132489d994e84b4edbb5b6f/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:02:29.565054 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b424e-316c-4048-ac9d-a2422adf958b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:29.576492 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b424e-316c-4048-ac9d-a2422adf958b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:29.632608 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:02:29.649967 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0073d2c-599a-4f2c-b215-8fdb8f958ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:29.650985 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0073d2c-599a-4f2c-b215-8fdb8f958ab7 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:29.651497 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0073d2c-599a-4f2c-b215-8fdb8f958ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:29.652077 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0073d2c-599a-4f2c-b215-8fdb8f958ab7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:29.652915 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0073d2c-599a-4f2c-b215-8fdb8f958ab7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:29.654582 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0073d2c-599a-4f2c-b215-8fdb8f958ab7 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:29.671582 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1282/2551] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:02:29.673177 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a06ba-0fff-41dd-b950-571149bdd3b6 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:29.690315 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a06ba-0fff-41dd-b950-571149bdd3b6 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:29.703388 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a5df6a-31be-4a32-9211-4837f1ba208b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:29.704121 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5df6a-31be-4a32-9211-4837f1ba208b admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:29.704408 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5df6a-31be-4a32-9211-4837f1ba208b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:29.707932 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5df6a-31be-4a32-9211-4837f1ba208b admin admin] PATH_INFO: `/v3/projects/6bed6a59cea946229b29e16bbe0900d8/users/3da20bf7a132489d994e84b4edbb5b6f/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:29.708148 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a5df6a-31be-4a32-9211-4837f1ba208b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bed6a59cea946229b29e16bbe0900d8, user_id=3da20bf7a132489d994e84b4edbb5b6f, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:29.711472 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a5df6a-31be-4a32-9211-4837f1ba208b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:29.726462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1270/2552] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:02:29 2022] PUT /identity/v3/projects/6bed6a59cea946229b29e16bbe0900d8/users/3da20bf7a132489d994e84b4edbb5b6f/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:02:29.739284 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6ccb0-5ea5-488b-9a5c-7e47fbb2a49e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:29.739880 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6ccb0-5ea5-488b-9a5c-7e47fbb2a49e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:29.740110 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6ccb0-5ea5-488b-9a5c-7e47fbb2a49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:29.777416 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-79e6ccb0-5ea5-488b-9a5c-7e47fbb2a49e None None] MFA Rules not processed for user `3da20bf7a132489d994e84b4edbb5b6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:02:29.779031 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-79e6ccb0-5ea5-488b-9a5c-7e47fbb2a49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:29.877033 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1283/2553] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:02:29 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:02:29.885736 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a5df6a-31be-4a32-9211-4837f1ba208b admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:29.888674 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-23a5df6a-31be-4a32-9211-4837f1ba208b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:29.906490 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:02:29.963121 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a5df6a-31be-4a32-9211-4837f1ba208b admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:29.970639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cac8bf-5fb1-4af2-8300-55239ce18b70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:29.971354 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cac8bf-5fb1-4af2-8300-55239ce18b70 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:29.971731 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cac8bf-5fb1-4af2-8300-55239ce18b70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:29.972082 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cac8bf-5fb1-4af2-8300-55239ce18b70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:29.983882 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cac8bf-5fb1-4af2-8300-55239ce18b70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:29.985643 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cac8bf-5fb1-4af2-8300-55239ce18b70 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:30.011984 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1271/2554] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:02:29 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:02:30.546240 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6ccb0-5ea5-488b-9a5c-7e47fbb2a49e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:30.558464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6ccb0-5ea5-488b-9a5c-7e47fbb2a49e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:30.562412 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4179a-bfaa-4fd9-b93f-87e145054b23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:30.563215 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4179a-bfaa-4fd9-b93f-87e145054b23 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:30.563593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4179a-bfaa-4fd9-b93f-87e145054b23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:30.563838 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4179a-bfaa-4fd9-b93f-87e145054b23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:30.636248 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e4179a-bfaa-4fd9-b93f-87e145054b23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:30.638053 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e4179a-bfaa-4fd9-b93f-87e145054b23 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:30.648646 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1284/2555] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:02:30 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:02:40.037075 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=23 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:02:40.644525 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=71 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:02:45.356090 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cac8bf-5fb1-4af2-8300-55239ce18b70 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:45.419546 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cac8bf-5fb1-4af2-8300-55239ce18b70 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:45.424153 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2da18-ce60-43f0-aa48-1a0002629e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:45.424799 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2da18-ce60-43f0-aa48-1a0002629e7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:45.425099 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2da18-ce60-43f0-aa48-1a0002629e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:45.425361 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2da18-ce60-43f0-aa48-1a0002629e7f admin admin] PATH_INFO: `/v3/users/2a74bb723b064c1b82c85eb878fab45f` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:45.426522 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f2da18-ce60-43f0-aa48-1a0002629e7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a74bb723b064c1b82c85eb878fab45f)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:45.428564 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f2da18-ce60-43f0-aa48-1a0002629e7f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:45.785010 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-49f2da18-ce60-43f0-aa48-1a0002629e7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a74bb723b064c1b82c85eb878fab45f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:45.785353 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-49f2da18-ce60-43f0-aa48-1a0002629e7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a74bb723b064c1b82c85eb878fab45f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:45.799594 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-49f2da18-ce60-43f0-aa48-1a0002629e7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a74bb723b064c1b82c85eb878fab45f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:45.807054 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-49f2da18-ce60-43f0-aa48-1a0002629e7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a74bb723b064c1b82c85eb878fab45f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:45.807922 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-49f2da18-ce60-43f0-aa48-1a0002629e7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a74bb723b064c1b82c85eb878fab45f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:45.967986 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1272/2556] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:02:45 2022] DELETE /identity/v3/users/2a74bb723b064c1b82c85eb878fab45f => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:02:45.972913 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b17061-9149-415e-991e-d2eb94d0e7ff None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:45.973463 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b17061-9149-415e-991e-d2eb94d0e7ff None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:45.973463 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b17061-9149-415e-991e-d2eb94d0e7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:45.998001 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c3b17061-9149-415e-991e-d2eb94d0e7ff None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:02:45.999877 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c3b17061-9149-415e-991e-d2eb94d0e7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:46.072841 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1285/2557] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:02:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:02:46.083140 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2da18-ce60-43f0-aa48-1a0002629e7f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:46.084634 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-49f2da18-ce60-43f0-aa48-1a0002629e7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:46.148917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2da18-ce60-43f0-aa48-1a0002629e7f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:46.152838 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e201bb5e-1d44-4a8a-9d1b-a223f4509883 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:46.153484 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201bb5e-1d44-4a8a-9d1b-a223f4509883 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:46.153777 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201bb5e-1d44-4a8a-9d1b-a223f4509883 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:46.154057 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201bb5e-1d44-4a8a-9d1b-a223f4509883 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:46.165103 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e201bb5e-1d44-4a8a-9d1b-a223f4509883 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:46.166573 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e201bb5e-1d44-4a8a-9d1b-a223f4509883 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:46.176524 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1273/2558] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:02:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:02:46.839826 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b17061-9149-415e-991e-d2eb94d0e7ff None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:46.841424 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c3b17061-9149-415e-991e-d2eb94d0e7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:46.855326 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b17061-9149-415e-991e-d2eb94d0e7ff None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:46.860023 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff1818-d969-44c4-b5b9-ca6a36c69400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:46.860977 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff1818-d969-44c4-b5b9-ca6a36c69400 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:46.860977 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff1818-d969-44c4-b5b9-ca6a36c69400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:46.861182 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff1818-d969-44c4-b5b9-ca6a36c69400 admin admin] PATH_INFO: `/v3/projects/fe5199415c124a4dae2e9f965e5f715f` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:46.862429 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff1818-d969-44c4-b5b9-ca6a36c69400 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe5199415c124a4dae2e9f965e5f715f)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:46.864391 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff1818-d969-44c4-b5b9-ca6a36c69400 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:47.021785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2cff1818-d969-44c4-b5b9-ca6a36c69400 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe5199415c124a4dae2e9f965e5f715f', 'request_id': 'req-2cff1818-d969-44c4-b5b9-ca6a36c69400'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:47.028437 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2cff1818-d969-44c4-b5b9-ca6a36c69400 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe5199415c124a4dae2e9f965e5f715f', 'request_id': 'req-2cff1818-d969-44c4-b5b9-ca6a36c69400'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:47.104422 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2cff1818-d969-44c4-b5b9-ca6a36c69400 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe5199415c124a4dae2e9f965e5f715f', 'request_id': 'req-2cff1818-d969-44c4-b5b9-ca6a36c69400'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:47.128549 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2cff1818-d969-44c4-b5b9-ca6a36c69400 admin admin] The token cache is being invalidate because project fe5199415c124a4dae2e9f965e5f715f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:02:47.128865 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2cff1818-d969-44c4-b5b9-ca6a36c69400 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:47.129527 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2cff1818-d969-44c4-b5b9-ca6a36c69400 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:47.132045 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1286/2559] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:02:46 2022] DELETE /identity/v3/projects/fe5199415c124a4dae2e9f965e5f715f => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:02:47.152874 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e201bb5e-1d44-4a8a-9d1b-a223f4509883 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:47.163454 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e201bb5e-1d44-4a8a-9d1b-a223f4509883 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:47.182541 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:02:47.222243 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b0696-48d6-4b4e-a514-2748752a9200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:47.222974 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b0696-48d6-4b4e-a514-2748752a9200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:47.223255 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b0696-48d6-4b4e-a514-2748752a9200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:47.223774 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b0696-48d6-4b4e-a514-2748752a9200 admin admin] PATH_INFO: `/v3/users/3da20bf7a132489d994e84b4edbb5b6f` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:47.225480 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6b0696-48d6-4b4e-a514-2748752a9200 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3da20bf7a132489d994e84b4edbb5b6f)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:47.227559 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6b0696-48d6-4b4e-a514-2748752a9200 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:47.537411 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9b6b0696-48d6-4b4e-a514-2748752a9200 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3da20bf7a132489d994e84b4edbb5b6f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:47.538075 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9b6b0696-48d6-4b4e-a514-2748752a9200 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3da20bf7a132489d994e84b4edbb5b6f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:47.549617 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9b6b0696-48d6-4b4e-a514-2748752a9200 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3da20bf7a132489d994e84b4edbb5b6f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:47.556851 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9b6b0696-48d6-4b4e-a514-2748752a9200 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3da20bf7a132489d994e84b4edbb5b6f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:47.557520 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9b6b0696-48d6-4b4e-a514-2748752a9200 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3da20bf7a132489d994e84b4edbb5b6f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:47.681455 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1274/2560] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:02:47 2022] DELETE /identity/v3/users/3da20bf7a132489d994e84b4edbb5b6f => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:02:48.218624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff1818-d969-44c4-b5b9-ca6a36c69400 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:48.220140 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2cff1818-d969-44c4-b5b9-ca6a36c69400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:48.235627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff1818-d969-44c4-b5b9-ca6a36c69400 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:48.294515 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae8ea92-50b0-4af9-8fb7-b43f68331883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:48.297369 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8ea92-50b0-4af9-8fb7-b43f68331883 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:48.297369 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8ea92-50b0-4af9-8fb7-b43f68331883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:48.297369 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8ea92-50b0-4af9-8fb7-b43f68331883 admin admin] PATH_INFO: `/v3/projects/6bed6a59cea946229b29e16bbe0900d8` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:48.297369 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae8ea92-50b0-4af9-8fb7-b43f68331883 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bed6a59cea946229b29e16bbe0900d8)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:48.298782 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae8ea92-50b0-4af9-8fb7-b43f68331883 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:48.392053 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:02:48.409246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8ae8ea92-50b0-4af9-8fb7-b43f68331883 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bed6a59cea946229b29e16bbe0900d8', 'request_id': 'req-8ae8ea92-50b0-4af9-8fb7-b43f68331883'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:48.414591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8ae8ea92-50b0-4af9-8fb7-b43f68331883 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bed6a59cea946229b29e16bbe0900d8', 'request_id': 'req-8ae8ea92-50b0-4af9-8fb7-b43f68331883'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:48.529681 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8ae8ea92-50b0-4af9-8fb7-b43f68331883 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bed6a59cea946229b29e16bbe0900d8', 'request_id': 'req-8ae8ea92-50b0-4af9-8fb7-b43f68331883'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:48.543802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8ae8ea92-50b0-4af9-8fb7-b43f68331883 admin admin] The token cache is being invalidate because project 6bed6a59cea946229b29e16bbe0900d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:02:48.544132 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8ae8ea92-50b0-4af9-8fb7-b43f68331883 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:48.544946 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8ae8ea92-50b0-4af9-8fb7-b43f68331883 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:02:48.547520 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1287/2561] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:02:48 2022] DELETE /identity/v3/projects/6bed6a59cea946229b29e16bbe0900d8 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:02:48.551316 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe27382-b622-402a-ad6f-831b2d3ad290 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:48.551814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe27382-b622-402a-ad6f-831b2d3ad290 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:48.551914 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe27382-b622-402a-ad6f-831b2d3ad290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:48.572194 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-5fe27382-b622-402a-ad6f-831b2d3ad290 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:02:48.573528 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5fe27382-b622-402a-ad6f-831b2d3ad290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:48.650228 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1275/2562] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:02:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:02:48.659310 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dfa905-e15c-4f27-8eb7-dfb4eb970564 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:48.659649 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dfa905-e15c-4f27-8eb7-dfb4eb970564 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:48.659928 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dfa905-e15c-4f27-8eb7-dfb4eb970564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:48.685391 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a8dfa905-e15c-4f27-8eb7-dfb4eb970564 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:02:48.686593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a8dfa905-e15c-4f27-8eb7-dfb4eb970564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:48.695172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1288/2563] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:02:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:02:48.699784 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe27382-b622-402a-ad6f-831b2d3ad290 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:48.714859 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe27382-b622-402a-ad6f-831b2d3ad290 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:48.719850 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c55de9-e93d-4aaa-8dac-9ea8c7c1df47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:48.720473 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c55de9-e93d-4aaa-8dac-9ea8c7c1df47 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:48.720752 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c55de9-e93d-4aaa-8dac-9ea8c7c1df47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:48.721054 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c55de9-e93d-4aaa-8dac-9ea8c7c1df47 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:48.723804 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c55de9-e93d-4aaa-8dac-9ea8c7c1df47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:02:48.723804 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c55de9-e93d-4aaa-8dac-9ea8c7c1df47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:48.723804 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c55de9-e93d-4aaa-8dac-9ea8c7c1df47 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:48.738189 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1276/2564] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 15:02:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:02:48.743233 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dfa905-e15c-4f27-8eb7-dfb4eb970564 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:48.753851 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dfa905-e15c-4f27-8eb7-dfb4eb970564 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:48.759999 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52ad33-e554-4a2b-a8d7-a46fe859fbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:48.760628 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52ad33-e554-4a2b-a8d7-a46fe859fbf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:48.760940 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52ad33-e554-4a2b-a8d7-a46fe859fbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:48.761229 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52ad33-e554-4a2b-a8d7-a46fe859fbf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:48.761906 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d52ad33-e554-4a2b-a8d7-a46fe859fbf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:48.763878 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d52ad33-e554-4a2b-a8d7-a46fe859fbf8 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:48.886041 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1289/2565] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 15:02:48 2022] POST /identity/v3/projects => generated 373 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:02:48.890369 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c55de9-e93d-4aaa-8dac-9ea8c7c1df47 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:48.901321 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c55de9-e93d-4aaa-8dac-9ea8c7c1df47 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:48.927349 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:02:48.961539 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd6bc4b-314b-49b2-8568-128ccdb9cb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:48.962163 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd6bc4b-314b-49b2-8568-128ccdb9cb32 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:48.962451 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd6bc4b-314b-49b2-8568-128ccdb9cb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:48.962758 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd6bc4b-314b-49b2-8568-128ccdb9cb32 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:48.963496 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd6bc4b-314b-49b2-8568-128ccdb9cb32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:48.965427 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd6bc4b-314b-49b2-8568-128ccdb9cb32 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:49.113615 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1277/2566] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 15:02:48 2022] POST /identity/v3/users => generated 287 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:02:49.119699 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52ad33-e554-4a2b-a8d7-a46fe859fbf8 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:49.131852 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666d506-7771-49f7-b159-822d923a2e4e None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:49.132213 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666d506-7771-49f7-b159-822d923a2e4e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:49.132424 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666d506-7771-49f7-b159-822d923a2e4e None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:49.132762 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52ad33-e554-4a2b-a8d7-a46fe859fbf8 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:49.134258 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1278/2567] 10.222.0.97 () {54 vars in 921 bytes} [Tue Jul 26 15:02:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 15:02:49.138427 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8acda17-c492-4d3d-b56a-870b948d2543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:49.138992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acda17-c492-4d3d-b56a-870b948d2543 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:49.139256 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acda17-c492-4d3d-b56a-870b948d2543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:49.139660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acda17-c492-4d3d-b56a-870b948d2543 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:49.144954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8acda17-c492-4d3d-b56a-870b948d2543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:49.145098 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723ff4e-0eb3-4cf6-b75b-991d9c73d161 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:49.145098 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723ff4e-0eb3-4cf6-b75b-991d9c73d161 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:49.145263 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8acda17-c492-4d3d-b56a-870b948d2543 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:49.145341 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723ff4e-0eb3-4cf6-b75b-991d9c73d161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:49.155314 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1290/2568] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:02:49.159421 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8acda17-c492-4d3d-b56a-870b948d2543 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:49.170521 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8acda17-c492-4d3d-b56a-870b948d2543 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:49.175175 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad8019-7261-479e-bc1a-a629998a67c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:49.175832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad8019-7261-479e-bc1a-a629998a67c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:49.176141 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad8019-7261-479e-bc1a-a629998a67c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:49.176433 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad8019-7261-479e-bc1a-a629998a67c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:49.177123 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad8019-7261-479e-bc1a-a629998a67c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:49.179338 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad8019-7261-479e-bc1a-a629998a67c2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:49.193078 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1291/2569] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:02:49.197400 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad8019-7261-479e-bc1a-a629998a67c2 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:49.199678 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9723ff4e-0eb3-4cf6-b75b-991d9c73d161 None None] MFA Rules not processed for user `266abef6cda5495b897170f92043aa03`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:02:49.201112 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9723ff4e-0eb3-4cf6-b75b-991d9c73d161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:49.208610 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad8019-7261-479e-bc1a-a629998a67c2 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:49.213672 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966956bb-56b9-41e9-9b8a-55789fca166c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:49.214348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966956bb-56b9-41e9-9b8a-55789fca166c admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:49.214636 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966956bb-56b9-41e9-9b8a-55789fca166c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:49.214912 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966956bb-56b9-41e9-9b8a-55789fca166c admin admin] PATH_INFO: `/v3/projects/ba1773de981b41bd829bc90d92f2f649/users/39d959e5dbd346708e87b496d344af28/roles/e033961a9363456a8510c34414dd839e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:49.231287 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966956bb-56b9-41e9-9b8a-55789fca166c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba1773de981b41bd829bc90d92f2f649, user_id=39d959e5dbd346708e87b496d344af28, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:49.234115 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966956bb-56b9-41e9-9b8a-55789fca166c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:49.276068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1279/2570] 10.222.0.97 () {58 vars in 1018 bytes} [Tue Jul 26 15:02:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:02:49.287282 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723ff4e-0eb3-4cf6-b75b-991d9c73d161 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:49.289232 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9723ff4e-0eb3-4cf6-b75b-991d9c73d161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:49.345087 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723ff4e-0eb3-4cf6-b75b-991d9c73d161 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:49.346541 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1292/2571] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:02:49 2022] PUT /identity/v3/projects/ba1773de981b41bd829bc90d92f2f649/users/39d959e5dbd346708e87b496d344af28/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:02:49.352297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7cac7a-d562-42ff-b360-50c5129d5649 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:49.352586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7cac7a-d562-42ff-b360-50c5129d5649 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:49.352884 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7cac7a-d562-42ff-b360-50c5129d5649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:49.372169 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-8a7cac7a-d562-42ff-b360-50c5129d5649 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:02:49.373392 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8a7cac7a-d562-42ff-b360-50c5129d5649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:49.401183 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89208fda-c14d-418b-a944-8d736b66fdda service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62adb1f5e9654927847e1717630ff6b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:49.401183 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89208fda-c14d-418b-a944-8d736b66fdda service glance] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:49.401183 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89208fda-c14d-418b-a944-8d736b66fdda service glance] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:49.401183 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89208fda-c14d-418b-a944-8d736b66fdda service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:49.406064 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:02:49.434435 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1293/2572] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:02:49.445868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7cac7a-d562-42ff-b360-50c5129d5649 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:49.447467 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8a7cac7a-d562-42ff-b360-50c5129d5649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:49.471718 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89208fda-c14d-418b-a944-8d736b66fdda service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:49.473270 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89208fda-c14d-418b-a944-8d736b66fdda service glance] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:49.484677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1280/2573] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:02:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:02:49.494483 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:02:49.508768 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7cac7a-d562-42ff-b360-50c5129d5649 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:49.513441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b28538-196a-446b-89a7-ccc64be78f8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:49.514064 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b28538-196a-446b-89a7-ccc64be78f8c service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:49.514739 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b28538-196a-446b-89a7-ccc64be78f8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:49.515020 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b28538-196a-446b-89a7-ccc64be78f8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:49.525971 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b28538-196a-446b-89a7-ccc64be78f8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:49.527528 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b28538-196a-446b-89a7-ccc64be78f8c service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:49.537515 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1294/2574] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:02:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:02:49.788528 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208988d2-1f24-472c-b115-3b4423a6b171 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:49.789156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208988d2-1f24-472c-b115-3b4423a6b171 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:49.789156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208988d2-1f24-472c-b115-3b4423a6b171 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:49.790708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1281/2575] 10.222.0.97 () {54 vars in 921 bytes} [Tue Jul 26 15:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 15:02:49.795886 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f30e7-62e9-4ee3-b093-1c79bdd29199 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:49.796185 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f30e7-62e9-4ee3-b093-1c79bdd29199 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:49.796489 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f30e7-62e9-4ee3-b093-1c79bdd29199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:49.840509 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-285f30e7-62e9-4ee3-b093-1c79bdd29199 None None] MFA Rules not processed for user `c81180fb4ff04c1186a21e43fb597a91`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:02:49.841793 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-285f30e7-62e9-4ee3-b093-1c79bdd29199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:49.848595 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1295/2576] 10.222.0.97 () {58 vars in 1018 bytes} [Tue Jul 26 15:02:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:02:49.864718 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208988d2-1f24-472c-b115-3b4423a6b171 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:49.876910 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208988d2-1f24-472c-b115-3b4423a6b171 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:49.880499 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a4378-830c-4953-8847-3fec9ac5d049 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:49.881104 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a4378-830c-4953-8847-3fec9ac5d049 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:49.881405 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a4378-830c-4953-8847-3fec9ac5d049 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:49.881679 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a4378-830c-4953-8847-3fec9ac5d049 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:49.888999 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a4378-830c-4953-8847-3fec9ac5d049 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:49.892745 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a4378-830c-4953-8847-3fec9ac5d049 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:49.902858 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1282/2577] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:02:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:02:50.038542 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a8447-b203-4609-9c65-2c4fe2dcce2b None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:50.038971 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a8447-b203-4609-9c65-2c4fe2dcce2b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:50.039325 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a8447-b203-4609-9c65-2c4fe2dcce2b None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:50.040851 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1296/2578] 10.222.0.97 () {54 vars in 921 bytes} [Tue Jul 26 15:02:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 15:02:50.048631 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fcda9-613a-4ed6-8d66-39198ed93b90 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:50.052180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fcda9-613a-4ed6-8d66-39198ed93b90 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:50.052180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fcda9-613a-4ed6-8d66-39198ed93b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:50.069706 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e77fcda9-613a-4ed6-8d66-39198ed93b90 None None] MFA Rules not processed for user `266abef6cda5495b897170f92043aa03`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:02:50.070974 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e77fcda9-613a-4ed6-8d66-39198ed93b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:50.080890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1283/2579] 10.222.0.97 () {58 vars in 1018 bytes} [Tue Jul 26 15:02:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:02:50.093535 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a8447-b203-4609-9c65-2c4fe2dcce2b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:50.107003 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a8447-b203-4609-9c65-2c4fe2dcce2b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:50.110638 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda1c68-07f3-4c09-9298-e694dbb2d1e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:50.115076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda1c68-07f3-4c09-9298-e694dbb2d1e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:50.115076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda1c68-07f3-4c09-9298-e694dbb2d1e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:50.115076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda1c68-07f3-4c09-9298-e694dbb2d1e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:50.123420 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eda1c68-07f3-4c09-9298-e694dbb2d1e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:50.124876 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eda1c68-07f3-4c09-9298-e694dbb2d1e9 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:50.135619 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1297/2580] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:02:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:02:51.216283 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b744c-cdc8-4e11-ac42-b246a70ce116 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:51.216841 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b744c-cdc8-4e11-ac42-b246a70ce116 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:51.216841 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b744c-cdc8-4e11-ac42-b246a70ce116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:51.246035 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-0c1b744c-cdc8-4e11-ac42-b246a70ce116 None None] MFA Rules not processed for user `39d959e5dbd346708e87b496d344af28`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:02:51.247277 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0c1b744c-cdc8-4e11-ac42-b246a70ce116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:51.322312 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1284/2581] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:02:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:02:51.333033 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda1c68-07f3-4c09-9298-e694dbb2d1e9 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:51.334453 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4eda1c68-07f3-4c09-9298-e694dbb2d1e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:51.398181 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda1c68-07f3-4c09-9298-e694dbb2d1e9 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:51.402538 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f6fc0-d7de-418e-a563-bd7fe669cf51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:51.403451 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f6fc0-d7de-418e-a563-bd7fe669cf51 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:51.403653 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f6fc0-d7de-418e-a563-bd7fe669cf51 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:51.403949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f6fc0-d7de-418e-a563-bd7fe669cf51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:51.416240 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f6fc0-d7de-418e-a563-bd7fe669cf51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:51.417692 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f6fc0-d7de-418e-a563-bd7fe669cf51 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:51.427771 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1298/2582] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:02:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:02:54.516317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c3407-adcd-42fc-9896-0325b0d8a502 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:54.516880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c3407-adcd-42fc-9896-0325b0d8a502 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:54.516880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c3407-adcd-42fc-9896-0325b0d8a502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:54.540112 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-807c3407-adcd-42fc-9896-0325b0d8a502 None None] MFA Rules not processed for user `39d959e5dbd346708e87b496d344af28`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:02:54.540112 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-807c3407-adcd-42fc-9896-0325b0d8a502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:54.545573 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1285/2583] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:02:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:02:55.905349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f6fc0-d7de-418e-a563-bd7fe669cf51 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:55.906876 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c32f6fc0-d7de-418e-a563-bd7fe669cf51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:55.920413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f6fc0-d7de-418e-a563-bd7fe669cf51 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:55.924400 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6b601-7f4e-42ae-af41-625e96b7984d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:55.925020 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6b601-7f4e-42ae-af41-625e96b7984d service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:55.925304 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6b601-7f4e-42ae-af41-625e96b7984d service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:55.925583 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6b601-7f4e-42ae-af41-625e96b7984d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:55.927422 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4cd6b601-7f4e-42ae-af41-625e96b7984d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:55.937934 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6b601-7f4e-42ae-af41-625e96b7984d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:55.939364 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6b601-7f4e-42ae-af41-625e96b7984d service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:55.953074 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1299/2584] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:02:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:02:58.230043 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e547d00-98bd-435a-a659-d037193082a7 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:58.235550 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e547d00-98bd-435a-a659-d037193082a7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:58.235550 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e547d00-98bd-435a-a659-d037193082a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:58.253334 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9e547d00-98bd-435a-a659-d037193082a7 None None] MFA Rules not processed for user `5179c3c4eaaa41f3beb79ba7f1993478`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:02:58.254493 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9e547d00-98bd-435a-a659-d037193082a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:58.312400 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1286/2585] 10.222.0.97 () {58 vars in 1009 bytes} [Tue Jul 26 15:02:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:02:58.324525 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6b601-7f4e-42ae-af41-625e96b7984d service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:58.325929 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4cd6b601-7f4e-42ae-af41-625e96b7984d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:58.398634 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6b601-7f4e-42ae-af41-625e96b7984d service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:58.404248 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cea642-1566-41dd-a405-b50d6c761ac0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85122c4052fd4ba19a2dae185a707caf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:58.407413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cea642-1566-41dd-a405-b50d6c761ac0 service swift] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:58.407413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cea642-1566-41dd-a405-b50d6c761ac0 service swift] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:58.407413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cea642-1566-41dd-a405-b50d6c761ac0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:58.413985 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cea642-1566-41dd-a405-b50d6c761ac0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:58.415585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cea642-1566-41dd-a405-b50d6c761ac0 service swift] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:58.423569 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1300/2586] 10.222.0.97 () {60 vars in 1384 bytes} [Tue Jul 26 15:02:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:03:08.304810 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:03:08.420413 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=87 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:03:21.104041 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e547d00-98bd-435a-a659-d037193082a7 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:21.105464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9e547d00-98bd-435a-a659-d037193082a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:21.117509 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e547d00-98bd-435a-a659-d037193082a7 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:21.121296 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29406b0-318e-4b2a-85cd-b88e0fc740d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:21.121919 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29406b0-318e-4b2a-85cd-b88e0fc740d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:21.122203 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29406b0-318e-4b2a-85cd-b88e0fc740d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:21.122508 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29406b0-318e-4b2a-85cd-b88e0fc740d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:21.124250 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b29406b0-318e-4b2a-85cd-b88e0fc740d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:21.205614 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29406b0-318e-4b2a-85cd-b88e0fc740d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:21.207061 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29406b0-318e-4b2a-85cd-b88e0fc740d8 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:21.226147 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1287/2587] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:03:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:03:21.326170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cea642-1566-41dd-a405-b50d6c761ac0 service swift] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:21.337412 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cea642-1566-41dd-a405-b50d6c761ac0 service swift] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:21.340958 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01708007-a636-4f18-97d4-f3b74c55be19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:21.341567 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01708007-a636-4f18-97d4-f3b74c55be19 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:21.341850 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01708007-a636-4f18-97d4-f3b74c55be19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:21.342120 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01708007-a636-4f18-97d4-f3b74c55be19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:21.343880 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-01708007-a636-4f18-97d4-f3b74c55be19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:21.357226 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01708007-a636-4f18-97d4-f3b74c55be19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:21.358647 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01708007-a636-4f18-97d4-f3b74c55be19 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:21.369170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1301/2588] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:03:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:03:25.781510 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29406b0-318e-4b2a-85cd-b88e0fc740d8 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:25.782686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b29406b0-318e-4b2a-85cd-b88e0fc740d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:25.796770 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29406b0-318e-4b2a-85cd-b88e0fc740d8 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:25.802134 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a9a51-2aca-4382-ae0c-ca921703585d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:25.802816 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a9a51-2aca-4382-ae0c-ca921703585d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:25.803098 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a9a51-2aca-4382-ae0c-ca921703585d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:25.803425 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a9a51-2aca-4382-ae0c-ca921703585d admin admin] PATH_INFO: `/v3/users/de3b593763e44e28be0f51187defa75d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:25.804631 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32a9a51-2aca-4382-ae0c-ca921703585d admin admin] RBAC: Authorizing `identity:delete_user(user_id=de3b593763e44e28be0f51187defa75d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:25.806558 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32a9a51-2aca-4382-ae0c-ca921703585d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:26.055656 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b32a9a51-2aca-4382-ae0c-ca921703585d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de3b593763e44e28be0f51187defa75d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:26.056603 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b32a9a51-2aca-4382-ae0c-ca921703585d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de3b593763e44e28be0f51187defa75d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:26.070736 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b32a9a51-2aca-4382-ae0c-ca921703585d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de3b593763e44e28be0f51187defa75d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:26.078196 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b32a9a51-2aca-4382-ae0c-ca921703585d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de3b593763e44e28be0f51187defa75d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:26.078914 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b32a9a51-2aca-4382-ae0c-ca921703585d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de3b593763e44e28be0f51187defa75d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:26.214585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1288/2589] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:03:25 2022] DELETE /identity/v3/users/de3b593763e44e28be0f51187defa75d => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:03:26.593651 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01708007-a636-4f18-97d4-f3b74c55be19 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:26.610465 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01708007-a636-4f18-97d4-f3b74c55be19 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:26.674195 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36aa3c3-36d0-4ebe-a4b4-29f127fc5d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:26.674911 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36aa3c3-36d0-4ebe-a4b4-29f127fc5d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:26.675210 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36aa3c3-36d0-4ebe-a4b4-29f127fc5d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:26.675541 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36aa3c3-36d0-4ebe-a4b4-29f127fc5d50 admin admin] PATH_INFO: `/v3/projects/ace4455a9e1b44e5bf7b04fd4abf81d7` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:26.676731 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36aa3c3-36d0-4ebe-a4b4-29f127fc5d50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ace4455a9e1b44e5bf7b04fd4abf81d7)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:26.678631 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36aa3c3-36d0-4ebe-a4b4-29f127fc5d50 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:26.823747 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f36aa3c3-36d0-4ebe-a4b4-29f127fc5d50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ace4455a9e1b44e5bf7b04fd4abf81d7', 'request_id': 'req-f36aa3c3-36d0-4ebe-a4b4-29f127fc5d50'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:26.829136 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f36aa3c3-36d0-4ebe-a4b4-29f127fc5d50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ace4455a9e1b44e5bf7b04fd4abf81d7', 'request_id': 'req-f36aa3c3-36d0-4ebe-a4b4-29f127fc5d50'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:26.950804 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f36aa3c3-36d0-4ebe-a4b4-29f127fc5d50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ace4455a9e1b44e5bf7b04fd4abf81d7', 'request_id': 'req-f36aa3c3-36d0-4ebe-a4b4-29f127fc5d50'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:26.963558 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f36aa3c3-36d0-4ebe-a4b4-29f127fc5d50 admin admin] The token cache is being invalidate because project ace4455a9e1b44e5bf7b04fd4abf81d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:03:26.963853 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f36aa3c3-36d0-4ebe-a4b4-29f127fc5d50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:26.964497 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f36aa3c3-36d0-4ebe-a4b4-29f127fc5d50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:26.966041 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1302/2590] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:03:26 2022] DELETE /identity/v3/projects/ace4455a9e1b44e5bf7b04fd4abf81d7 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:03:26.970964 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908854b-45df-4ce8-b822-0eeb51035b82 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:26.971488 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908854b-45df-4ce8-b822-0eeb51035b82 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:26.971592 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908854b-45df-4ce8-b822-0eeb51035b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:26.992215 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-2908854b-45df-4ce8-b822-0eeb51035b82 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:03:26.993445 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2908854b-45df-4ce8-b822-0eeb51035b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:27.035607 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:03:27.061180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1289/2591] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:03:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:03:27.069711 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d612d-19d1-4356-ab35-c25fed378f43 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:27.070002 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d612d-19d1-4356-ab35-c25fed378f43 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:27.070287 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d612d-19d1-4356-ab35-c25fed378f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:27.090877 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f67d612d-19d1-4356-ab35-c25fed378f43 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:03:27.092128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f67d612d-19d1-4356-ab35-c25fed378f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:27.100718 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1303/2592] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:03:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:03:27.105257 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2908854b-45df-4ce8-b822-0eeb51035b82 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:27.115284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2908854b-45df-4ce8-b822-0eeb51035b82 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:27.120853 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb25fd-bc16-4dc6-962b-cc6e8fb02225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:27.121462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb25fd-bc16-4dc6-962b-cc6e8fb02225 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:27.121940 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb25fd-bc16-4dc6-962b-cc6e8fb02225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:27.122229 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb25fd-bc16-4dc6-962b-cc6e8fb02225 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:27.123004 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb25fd-bc16-4dc6-962b-cc6e8fb02225 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:03:27.123244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb25fd-bc16-4dc6-962b-cc6e8fb02225 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:27.124877 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb25fd-bc16-4dc6-962b-cc6e8fb02225 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:27.136778 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1290/2593] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 15:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:03:27.141214 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d612d-19d1-4356-ab35-c25fed378f43 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:27.151440 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d612d-19d1-4356-ab35-c25fed378f43 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:27.156623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60083d52-bff7-4f4b-8209-f233f2dfcbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:27.168070 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60083d52-bff7-4f4b-8209-f233f2dfcbea admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:27.168070 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60083d52-bff7-4f4b-8209-f233f2dfcbea admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:27.168070 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60083d52-bff7-4f4b-8209-f233f2dfcbea admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:27.168070 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60083d52-bff7-4f4b-8209-f233f2dfcbea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:27.168070 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60083d52-bff7-4f4b-8209-f233f2dfcbea admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:27.353014 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1304/2594] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 15:03:27 2022] POST /identity/v3/projects => generated 403 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:03:27.357116 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb25fd-bc16-4dc6-962b-cc6e8fb02225 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:27.367643 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb25fd-bc16-4dc6-962b-cc6e8fb02225 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:27.427375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f91eeba-880f-4858-be44-e178c6908c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:27.427375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91eeba-880f-4858-be44-e178c6908c05 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:27.428063 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91eeba-880f-4858-be44-e178c6908c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:27.428561 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91eeba-880f-4858-be44-e178c6908c05 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:27.429624 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f91eeba-880f-4858-be44-e178c6908c05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:27.432020 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f91eeba-880f-4858-be44-e178c6908c05 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:27.459674 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cad6a0-2574-402a-a34a-c5004eefb9c3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:27.460027 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cad6a0-2574-402a-a34a-c5004eefb9c3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:27.460373 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cad6a0-2574-402a-a34a-c5004eefb9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:27.481844 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-39cad6a0-2574-402a-a34a-c5004eefb9c3 None None] MFA Rules not processed for user `5179c3c4eaaa41f3beb79ba7f1993478`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:03:27.483105 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-39cad6a0-2574-402a-a34a-c5004eefb9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:27.523692 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1291/2595] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 15:03:27 2022] POST /identity/v3/users => generated 302 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:03:27.527738 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f91eeba-880f-4858-be44-e178c6908c05 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:27.532401 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:03:27.541590 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f91eeba-880f-4858-be44-e178c6908c05 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:27.546035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1305/2596] 10.222.0.97 () {58 vars in 967 bytes} [Tue Jul 26 15:03:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:03:27.547613 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d8f1b-c46c-4496-9ce9-390878b027a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:27.548270 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d8f1b-c46c-4496-9ce9-390878b027a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:27.548569 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d8f1b-c46c-4496-9ce9-390878b027a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:27.548826 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d8f1b-c46c-4496-9ce9-390878b027a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:27.549535 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786d8f1b-c46c-4496-9ce9-390878b027a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:27.551135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786d8f1b-c46c-4496-9ce9-390878b027a8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:27.562866 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1292/2597] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:03:27.570504 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cad6a0-2574-402a-a34a-c5004eefb9c3 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:27.574609 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d8f1b-c46c-4496-9ce9-390878b027a8 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:27.576686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-786d8f1b-c46c-4496-9ce9-390878b027a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:27.582675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cad6a0-2574-402a-a34a-c5004eefb9c3 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:27.589448 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d6396-11e0-40a1-9228-cf95463e74c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:27.590082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d6396-11e0-40a1-9228-cf95463e74c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:27.590360 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d6396-11e0-40a1-9228-cf95463e74c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:27.590644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d6396-11e0-40a1-9228-cf95463e74c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:27.591345 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d6396-11e0-40a1-9228-cf95463e74c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:27.593484 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d6396-11e0-40a1-9228-cf95463e74c8 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:27.609070 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1306/2598] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:03:27.610014 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d6396-11e0-40a1-9228-cf95463e74c8 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:27.620821 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d6396-11e0-40a1-9228-cf95463e74c8 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:27.628038 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b5c8bd-ac20-4674-8f8e-614b61196e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:27.628291 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5c8bd-ac20-4674-8f8e-614b61196e31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:27.628585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5c8bd-ac20-4674-8f8e-614b61196e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:27.628868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5c8bd-ac20-4674-8f8e-614b61196e31 admin admin] PATH_INFO: `/v3/projects/e5f0dc08c2c046f9901d5c265a0b1a3f/users/1b66d49640d74181951c57621cc25d87/roles/e033961a9363456a8510c34414dd839e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:27.645400 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b5c8bd-ac20-4674-8f8e-614b61196e31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5f0dc08c2c046f9901d5c265a0b1a3f, user_id=1b66d49640d74181951c57621cc25d87, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:27.648899 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b5c8bd-ac20-4674-8f8e-614b61196e31 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:27.662537 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d8f1b-c46c-4496-9ce9-390878b027a8 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:27.667625 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f05143a-0b53-4d92-99f1-b790a5d58c3f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85122c4052fd4ba19a2dae185a707caf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:27.668241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05143a-0b53-4d92-99f1-b790a5d58c3f service swift] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:27.668522 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05143a-0b53-4d92-99f1-b790a5d58c3f service swift] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:27.668796 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05143a-0b53-4d92-99f1-b790a5d58c3f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:27.676964 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f05143a-0b53-4d92-99f1-b790a5d58c3f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:27.678626 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f05143a-0b53-4d92-99f1-b790a5d58c3f service swift] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:27.687074 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1293/2599] 10.222.0.97 () {60 vars in 1384 bytes} [Tue Jul 26 15:03:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 15:03:27.756048 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1307/2600] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:03:27 2022] PUT /identity/v3/projects/e5f0dc08c2c046f9901d5c265a0b1a3f/users/1b66d49640d74181951c57621cc25d87/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:03:27.767635 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba1281-d550-4b83-a48b-67ab6003b97f None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:27.767951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba1281-d550-4b83-a48b-67ab6003b97f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:27.768259 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba1281-d550-4b83-a48b-67ab6003b97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:27.796229 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-daba1281-d550-4b83-a48b-67ab6003b97f None None] MFA Rules not processed for user `1b66d49640d74181951c57621cc25d87`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:03:27.797433 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-daba1281-d550-4b83-a48b-67ab6003b97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:27.825699 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:03:27.865915 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1294/2601] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:03:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:03:27.870386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b5c8bd-ac20-4674-8f8e-614b61196e31 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:27.933890 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b5c8bd-ac20-4674-8f8e-614b61196e31 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:27.938614 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc5acd4-20a6-4466-bd18-4e18f7d757f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:27.939257 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5acd4-20a6-4466-bd18-4e18f7d757f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:27.939578 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5acd4-20a6-4466-bd18-4e18f7d757f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:27.939854 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5acd4-20a6-4466-bd18-4e18f7d757f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:27.940510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc5acd4-20a6-4466-bd18-4e18f7d757f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:27.943163 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc5acd4-20a6-4466-bd18-4e18f7d757f4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:28.084304 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1308/2602] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 15:03:27 2022] POST /identity/v3/projects => generated 405 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:03:28.088454 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba1281-d550-4b83-a48b-67ab6003b97f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:28.099583 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba1281-d550-4b83-a48b-67ab6003b97f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:28.160040 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a713c-c788-4b45-b234-bec90d0d1969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:28.160664 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a713c-c788-4b45-b234-bec90d0d1969 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:28.161469 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a713c-c788-4b45-b234-bec90d0d1969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:28.161747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a713c-c788-4b45-b234-bec90d0d1969 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:28.162410 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a713c-c788-4b45-b234-bec90d0d1969 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:28.164396 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a713c-c788-4b45-b234-bec90d0d1969 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:28.292892 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1295/2603] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 15:03:28 2022] POST /identity/v3/users => generated 309 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:03:28.297817 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc5acd4-20a6-4466-bd18-4e18f7d757f4 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:28.307001 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc5acd4-20a6-4466-bd18-4e18f7d757f4 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:28.311877 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff3c20-fbd7-48f4-ad64-175ff2b5e3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:28.312498 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff3c20-fbd7-48f4-ad64-175ff2b5e3b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:28.312851 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff3c20-fbd7-48f4-ad64-175ff2b5e3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:28.313134 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff3c20-fbd7-48f4-ad64-175ff2b5e3b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:28.313834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff3c20-fbd7-48f4-ad64-175ff2b5e3b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:28.315441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff3c20-fbd7-48f4-ad64-175ff2b5e3b6 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:28.328249 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1309/2604] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:03:28.332584 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a713c-c788-4b45-b234-bec90d0d1969 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:28.343282 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a713c-c788-4b45-b234-bec90d0d1969 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:28.350134 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c169f0-6f12-4b42-8729-c2bbc0680b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:28.350821 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c169f0-6f12-4b42-8729-c2bbc0680b1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:28.351107 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c169f0-6f12-4b42-8729-c2bbc0680b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:28.351387 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c169f0-6f12-4b42-8729-c2bbc0680b1d admin admin] PATH_INFO: `/v3/projects/0045003d427a4fe9bccf5ae2e917cfa1/users/7e4f165e14ae4498b9f53dcc39512ce0/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:28.368274 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c169f0-6f12-4b42-8729-c2bbc0680b1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0045003d427a4fe9bccf5ae2e917cfa1, user_id=7e4f165e14ae4498b9f53dcc39512ce0, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:28.373664 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c169f0-6f12-4b42-8729-c2bbc0680b1d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:28.465842 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1296/2605] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:03:28 2022] PUT /identity/v3/projects/0045003d427a4fe9bccf5ae2e917cfa1/users/7e4f165e14ae4498b9f53dcc39512ce0/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:03:28.472202 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff3c20-fbd7-48f4-ad64-175ff2b5e3b6 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:28.482420 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff3c20-fbd7-48f4-ad64-175ff2b5e3b6 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:28.541315 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5b6fd-e2d5-4567-8390-fa8cb7939373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:28.541941 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5b6fd-e2d5-4567-8390-fa8cb7939373 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:28.542224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5b6fd-e2d5-4567-8390-fa8cb7939373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:28.542501 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5b6fd-e2d5-4567-8390-fa8cb7939373 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:28.543192 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5b6fd-e2d5-4567-8390-fa8cb7939373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:28.544978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5b6fd-e2d5-4567-8390-fa8cb7939373 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:28.556412 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1310/2606] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:03:28.560853 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c169f0-6f12-4b42-8729-c2bbc0680b1d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:28.571414 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c169f0-6f12-4b42-8729-c2bbc0680b1d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:28.576231 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481bbae-f8d9-4a12-aa99-5fb517800761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:28.576892 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481bbae-f8d9-4a12-aa99-5fb517800761 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:28.577175 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481bbae-f8d9-4a12-aa99-5fb517800761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:28.577449 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481bbae-f8d9-4a12-aa99-5fb517800761 admin admin] PATH_INFO: `/v3/projects/0045003d427a4fe9bccf5ae2e917cfa1/users/7e4f165e14ae4498b9f53dcc39512ce0/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:28.579458 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c481bbae-f8d9-4a12-aa99-5fb517800761 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0045003d427a4fe9bccf5ae2e917cfa1, user_id=7e4f165e14ae4498b9f53dcc39512ce0, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:28.582102 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c481bbae-f8d9-4a12-aa99-5fb517800761 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:28.592984 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1297/2607] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:03:28 2022] PUT /identity/v3/projects/0045003d427a4fe9bccf5ae2e917cfa1/users/7e4f165e14ae4498b9f53dcc39512ce0/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:03:28.603931 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fb2ae-eca4-4b79-a0cf-d3de0162cf59 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:28.604222 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fb2ae-eca4-4b79-a0cf-d3de0162cf59 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:28.604503 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fb2ae-eca4-4b79-a0cf-d3de0162cf59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:28.633624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3f4fb2ae-eca4-4b79-a0cf-d3de0162cf59 None None] MFA Rules not processed for user `7e4f165e14ae4498b9f53dcc39512ce0`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:03:28.634826 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3f4fb2ae-eca4-4b79-a0cf-d3de0162cf59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:28.653191 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:03:28.707054 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1311/2608] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:03:28 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:03:28.719049 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481bbae-f8d9-4a12-aa99-5fb517800761 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:28.720854 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c481bbae-f8d9-4a12-aa99-5fb517800761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:28.784589 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481bbae-f8d9-4a12-aa99-5fb517800761 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:28.789390 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41002a0f-2d65-4885-a82d-c37e2a4c790d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:28.790295 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41002a0f-2d65-4885-a82d-c37e2a4c790d service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:28.790695 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41002a0f-2d65-4885-a82d-c37e2a4c790d service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:28.791154 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41002a0f-2d65-4885-a82d-c37e2a4c790d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:28.802930 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41002a0f-2d65-4885-a82d-c37e2a4c790d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:28.804233 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41002a0f-2d65-4885-a82d-c37e2a4c790d service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:28.822448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1298/2609] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:03:28 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:03:28.972395 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4fb2ae-eca4-4b79-a0cf-d3de0162cf59 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:29.034623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4fb2ae-eca4-4b79-a0cf-d3de0162cf59 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:29.039202 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8062b2cd-a3e3-43b2-8781-15a4a10ca13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:29.039868 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062b2cd-a3e3-43b2-8781-15a4a10ca13e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:29.040150 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062b2cd-a3e3-43b2-8781-15a4a10ca13e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:29.040450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062b2cd-a3e3-43b2-8781-15a4a10ca13e admin admin] PATH_INFO: `/v3/users/1b66d49640d74181951c57621cc25d87` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:29.041661 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8062b2cd-a3e3-43b2-8781-15a4a10ca13e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b66d49640d74181951c57621cc25d87)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:29.043596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8062b2cd-a3e3-43b2-8781-15a4a10ca13e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:29.365720 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8062b2cd-a3e3-43b2-8781-15a4a10ca13e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b66d49640d74181951c57621cc25d87'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:29.366498 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8062b2cd-a3e3-43b2-8781-15a4a10ca13e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b66d49640d74181951c57621cc25d87'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:29.379026 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8062b2cd-a3e3-43b2-8781-15a4a10ca13e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b66d49640d74181951c57621cc25d87'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:29.386881 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8062b2cd-a3e3-43b2-8781-15a4a10ca13e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b66d49640d74181951c57621cc25d87'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:29.387730 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8062b2cd-a3e3-43b2-8781-15a4a10ca13e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b66d49640d74181951c57621cc25d87'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:29.507787 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1312/2610] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:03:28 2022] DELETE /identity/v3/users/1b66d49640d74181951c57621cc25d87 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:03:29.512208 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155a76f-bf6c-46a6-829f-7a7b90ada3d7 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:29.512724 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155a76f-bf6c-46a6-829f-7a7b90ada3d7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:29.512835 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155a76f-bf6c-46a6-829f-7a7b90ada3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:29.532152 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-4155a76f-bf6c-46a6-829f-7a7b90ada3d7 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:03:29.533410 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4155a76f-bf6c-46a6-829f-7a7b90ada3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:29.550714 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:03:29.597255 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1299/2611] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:03:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:03:29.607812 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8062b2cd-a3e3-43b2-8781-15a4a10ca13e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:29.609386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8062b2cd-a3e3-43b2-8781-15a4a10ca13e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:29.649469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:03:29.673499 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8062b2cd-a3e3-43b2-8781-15a4a10ca13e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:29.677674 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f832a249-4ece-4bf0-8d01-e4e09983a4aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:29.678291 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f832a249-4ece-4bf0-8d01-e4e09983a4aa service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:29.678569 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f832a249-4ece-4bf0-8d01-e4e09983a4aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:29.678849 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f832a249-4ece-4bf0-8d01-e4e09983a4aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:29.688364 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f832a249-4ece-4bf0-8d01-e4e09983a4aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:29.689817 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f832a249-4ece-4bf0-8d01-e4e09983a4aa service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:29.700590 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1313/2612] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:03:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:03:29.749144 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155a76f-bf6c-46a6-829f-7a7b90ada3d7 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:29.750707 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4155a76f-bf6c-46a6-829f-7a7b90ada3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:29.828429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155a76f-bf6c-46a6-829f-7a7b90ada3d7 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:29.834259 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c07ea3-79ef-468b-a2b6-02690cc837bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:29.834891 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c07ea3-79ef-468b-a2b6-02690cc837bb service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:29.835182 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c07ea3-79ef-468b-a2b6-02690cc837bb service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:29.835503 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c07ea3-79ef-468b-a2b6-02690cc837bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:29.837302 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-82c07ea3-79ef-468b-a2b6-02690cc837bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:29.850377 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c07ea3-79ef-468b-a2b6-02690cc837bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:29.854631 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c07ea3-79ef-468b-a2b6-02690cc837bb service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:29.863816 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1300/2613] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:03:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:03:30.181735 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f832a249-4ece-4bf0-8d01-e4e09983a4aa service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:30.183254 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f832a249-4ece-4bf0-8d01-e4e09983a4aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:30.197054 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f832a249-4ece-4bf0-8d01-e4e09983a4aa service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:30.202774 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad69b19-0822-4007-aa81-8ff6cf403bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:30.203805 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad69b19-0822-4007-aa81-8ff6cf403bc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:30.204089 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad69b19-0822-4007-aa81-8ff6cf403bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:30.204367 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad69b19-0822-4007-aa81-8ff6cf403bc6 admin admin] PATH_INFO: `/v3/projects/e5f0dc08c2c046f9901d5c265a0b1a3f` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:30.205551 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad69b19-0822-4007-aa81-8ff6cf403bc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5f0dc08c2c046f9901d5c265a0b1a3f)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:30.208092 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad69b19-0822-4007-aa81-8ff6cf403bc6 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:30.361866 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7ad69b19-0822-4007-aa81-8ff6cf403bc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5f0dc08c2c046f9901d5c265a0b1a3f', 'request_id': 'req-7ad69b19-0822-4007-aa81-8ff6cf403bc6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:30.367831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7ad69b19-0822-4007-aa81-8ff6cf403bc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5f0dc08c2c046f9901d5c265a0b1a3f', 'request_id': 'req-7ad69b19-0822-4007-aa81-8ff6cf403bc6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:30.448966 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7ad69b19-0822-4007-aa81-8ff6cf403bc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5f0dc08c2c046f9901d5c265a0b1a3f', 'request_id': 'req-7ad69b19-0822-4007-aa81-8ff6cf403bc6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:30.462081 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7ad69b19-0822-4007-aa81-8ff6cf403bc6 admin admin] The token cache is being invalidate because project e5f0dc08c2c046f9901d5c265a0b1a3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:03:30.462482 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7ad69b19-0822-4007-aa81-8ff6cf403bc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:30.463445 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7ad69b19-0822-4007-aa81-8ff6cf403bc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:30.466534 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1314/2614] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:03:30 2022] DELETE /identity/v3/projects/e5f0dc08c2c046f9901d5c265a0b1a3f => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:03:30.469324 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c07ea3-79ef-468b-a2b6-02690cc837bb service nova] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:30.480443 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c07ea3-79ef-468b-a2b6-02690cc837bb service nova] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:30.540870 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fed6d5-02d6-4487-84d9-0c4742600bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:30.541500 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fed6d5-02d6-4487-84d9-0c4742600bc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:30.541789 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fed6d5-02d6-4487-84d9-0c4742600bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:30.542063 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fed6d5-02d6-4487-84d9-0c4742600bc3 admin admin] PATH_INFO: `/v3/users/7e4f165e14ae4498b9f53dcc39512ce0` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:30.543297 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fed6d5-02d6-4487-84d9-0c4742600bc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e4f165e14ae4498b9f53dcc39512ce0)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:30.545401 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fed6d5-02d6-4487-84d9-0c4742600bc3 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:30.774406 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f9fed6d5-02d6-4487-84d9-0c4742600bc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e4f165e14ae4498b9f53dcc39512ce0'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:30.775113 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f9fed6d5-02d6-4487-84d9-0c4742600bc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e4f165e14ae4498b9f53dcc39512ce0'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:30.777247 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:03:30.787547 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f9fed6d5-02d6-4487-84d9-0c4742600bc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e4f165e14ae4498b9f53dcc39512ce0'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:30.794471 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f9fed6d5-02d6-4487-84d9-0c4742600bc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e4f165e14ae4498b9f53dcc39512ce0'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:30.795176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f9fed6d5-02d6-4487-84d9-0c4742600bc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e4f165e14ae4498b9f53dcc39512ce0'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:30.945864 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1301/2615] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:03:30 2022] DELETE /identity/v3/users/7e4f165e14ae4498b9f53dcc39512ce0 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:03:31.396432 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad69b19-0822-4007-aa81-8ff6cf403bc6 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:31.398185 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7ad69b19-0822-4007-aa81-8ff6cf403bc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:31.411701 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad69b19-0822-4007-aa81-8ff6cf403bc6 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:31.473837 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb9476-8f15-44a8-9fe8-870c568eb672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:31.474482 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb9476-8f15-44a8-9fe8-870c568eb672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:31.474771 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb9476-8f15-44a8-9fe8-870c568eb672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:31.475054 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb9476-8f15-44a8-9fe8-870c568eb672 admin admin] PATH_INFO: `/v3/projects/0045003d427a4fe9bccf5ae2e917cfa1` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:31.476256 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb9476-8f15-44a8-9fe8-870c568eb672 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0045003d427a4fe9bccf5ae2e917cfa1)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:31.478201 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb9476-8f15-44a8-9fe8-870c568eb672 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:31.591879 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f7bb9476-8f15-44a8-9fe8-870c568eb672 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0045003d427a4fe9bccf5ae2e917cfa1', 'request_id': 'req-f7bb9476-8f15-44a8-9fe8-870c568eb672'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:31.597291 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f7bb9476-8f15-44a8-9fe8-870c568eb672 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0045003d427a4fe9bccf5ae2e917cfa1', 'request_id': 'req-f7bb9476-8f15-44a8-9fe8-870c568eb672'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:31.600530 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:03:31.716414 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f7bb9476-8f15-44a8-9fe8-870c568eb672 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0045003d427a4fe9bccf5ae2e917cfa1', 'request_id': 'req-f7bb9476-8f15-44a8-9fe8-870c568eb672'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:31.728175 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f7bb9476-8f15-44a8-9fe8-870c568eb672 admin admin] The token cache is being invalidate because project 0045003d427a4fe9bccf5ae2e917cfa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:03:31.728402 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f7bb9476-8f15-44a8-9fe8-870c568eb672 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:31.729052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f7bb9476-8f15-44a8-9fe8-870c568eb672 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:31.730566 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1315/2616] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:03:31 2022] DELETE /identity/v3/projects/0045003d427a4fe9bccf5ae2e917cfa1 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:03:31.736372 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388bdd2-6faa-48f3-84da-9d212ffdc310 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:31.740917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388bdd2-6faa-48f3-84da-9d212ffdc310 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:31.740917 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388bdd2-6faa-48f3-84da-9d212ffdc310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:31.759951 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-8388bdd2-6faa-48f3-84da-9d212ffdc310 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:03:31.761229 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8388bdd2-6faa-48f3-84da-9d212ffdc310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:31.819948 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75c9eb-f662-44ec-8e5a-51fe0942ad88 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:31.820245 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75c9eb-f662-44ec-8e5a-51fe0942ad88 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:31.820511 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75c9eb-f662-44ec-8e5a-51fe0942ad88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:31.825856 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1302/2617] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:03:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:03:31.833695 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d19717-95ab-4246-b23a-8cc3d36ae136 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:31.833984 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d19717-95ab-4246-b23a-8cc3d36ae136 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:31.834272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d19717-95ab-4246-b23a-8cc3d36ae136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:31.842016 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-5b75c9eb-f662-44ec-8e5a-51fe0942ad88 None None] MFA Rules not processed for user `39d959e5dbd346708e87b496d344af28`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:03:31.843150 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5b75c9eb-f662-44ec-8e5a-51fe0942ad88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:31.854201 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-97d19717-95ab-4246-b23a-8cc3d36ae136 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:03:31.855505 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-97d19717-95ab-4246-b23a-8cc3d36ae136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:31.863483 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1303/2618] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:03:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:03:31.868156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d19717-95ab-4246-b23a-8cc3d36ae136 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:31.877029 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d19717-95ab-4246-b23a-8cc3d36ae136 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:31.881889 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d44b35-a7e7-4a03-bdc0-c44d0aa5c2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:31.882491 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d44b35-a7e7-4a03-bdc0-c44d0aa5c2bc admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:31.882775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d44b35-a7e7-4a03-bdc0-c44d0aa5c2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:31.883067 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d44b35-a7e7-4a03-bdc0-c44d0aa5c2bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:31.883857 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d44b35-a7e7-4a03-bdc0-c44d0aa5c2bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:03:31.884303 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d44b35-a7e7-4a03-bdc0-c44d0aa5c2bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:31.885967 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d44b35-a7e7-4a03-bdc0-c44d0aa5c2bc admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:31.898469 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1304/2619] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 15:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:03:31.902660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d44b35-a7e7-4a03-bdc0-c44d0aa5c2bc admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:31.905890 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1316/2620] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:03:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:03:31.914610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d44b35-a7e7-4a03-bdc0-c44d0aa5c2bc admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:31.919275 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9621bf0-8569-4ba0-a669-73fa14fcdb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:31.919950 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9621bf0-8569-4ba0-a669-73fa14fcdb2b admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:31.920259 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9621bf0-8569-4ba0-a669-73fa14fcdb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:31.920512 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9621bf0-8569-4ba0-a669-73fa14fcdb2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:31.921225 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9621bf0-8569-4ba0-a669-73fa14fcdb2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:31.923157 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9621bf0-8569-4ba0-a669-73fa14fcdb2b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:32.034795 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1305/2621] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 15:03:31 2022] POST /identity/v3/projects => generated 401 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:03:32.037054 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75c9eb-f662-44ec-8e5a-51fe0942ad88 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:32.049477 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75c9eb-f662-44ec-8e5a-51fe0942ad88 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:32.120765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274bded-e265-4345-875a-98911a70b972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:32.121404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274bded-e265-4345-875a-98911a70b972 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:32.121690 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274bded-e265-4345-875a-98911a70b972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:32.121964 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274bded-e265-4345-875a-98911a70b972 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:32.122636 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e274bded-e265-4345-875a-98911a70b972 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:32.124684 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e274bded-e265-4345-875a-98911a70b972 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:32.254963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1317/2622] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 15:03:32 2022] POST /identity/v3/users => generated 301 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:03:32.259138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9621bf0-8569-4ba0-a669-73fa14fcdb2b admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:32.269563 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9621bf0-8569-4ba0-a669-73fa14fcdb2b admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:32.274988 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0cee9d-a0c7-498f-b458-da046ff74d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:32.275655 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cee9d-a0c7-498f-b458-da046ff74d64 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:32.275938 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cee9d-a0c7-498f-b458-da046ff74d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:32.276214 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cee9d-a0c7-498f-b458-da046ff74d64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:32.276932 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0cee9d-a0c7-498f-b458-da046ff74d64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:32.278544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0cee9d-a0c7-498f-b458-da046ff74d64 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:32.289679 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1306/2623] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:03:32.293843 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274bded-e265-4345-875a-98911a70b972 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:32.304003 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274bded-e265-4345-875a-98911a70b972 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:32.308785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c840a4-6573-4732-ba98-5d6ea6b5f243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:32.309402 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c840a4-6573-4732-ba98-5d6ea6b5f243 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:32.309692 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c840a4-6573-4732-ba98-5d6ea6b5f243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:32.309967 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c840a4-6573-4732-ba98-5d6ea6b5f243 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:32.310690 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c840a4-6573-4732-ba98-5d6ea6b5f243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:32.312342 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c840a4-6573-4732-ba98-5d6ea6b5f243 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:32.322970 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1318/2624] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:03:32.327050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0cee9d-a0c7-498f-b458-da046ff74d64 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:32.337651 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0cee9d-a0c7-498f-b458-da046ff74d64 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:32.342330 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe5cff-abf0-4e3d-a649-e06fe06738dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:32.343000 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe5cff-abf0-4e3d-a649-e06fe06738dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:32.343287 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe5cff-abf0-4e3d-a649-e06fe06738dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:32.343596 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe5cff-abf0-4e3d-a649-e06fe06738dd admin admin] PATH_INFO: `/v3/projects/be3967716e0e45c3b691af0e01e04d0c/users/b6318b13ac6842b5b31d4e1afa2ade4b/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:32.360293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fe5cff-abf0-4e3d-a649-e06fe06738dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=be3967716e0e45c3b691af0e01e04d0c, user_id=b6318b13ac6842b5b31d4e1afa2ade4b, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:32.363298 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fe5cff-abf0-4e3d-a649-e06fe06738dd admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:32.441361 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1307/2625] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:03:32 2022] PUT /identity/v3/projects/be3967716e0e45c3b691af0e01e04d0c/users/b6318b13ac6842b5b31d4e1afa2ade4b/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:03:32.453845 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fdf51-2789-40a6-95bc-7ab94ef541a4 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:32.453845 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fdf51-2789-40a6-95bc-7ab94ef541a4 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:32.454493 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fdf51-2789-40a6-95bc-7ab94ef541a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:32.485450 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-866fdf51-2789-40a6-95bc-7ab94ef541a4 None None] MFA Rules not processed for user `b6318b13ac6842b5b31d4e1afa2ade4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:03:32.487866 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-866fdf51-2789-40a6-95bc-7ab94ef541a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:32.555988 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1319/2626] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:03:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:03:32.560897 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe5cff-abf0-4e3d-a649-e06fe06738dd admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:32.617367 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:03:32.630383 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe5cff-abf0-4e3d-a649-e06fe06738dd admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:32.635940 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954cc2d-5af0-4d50-89af-4cb8a8b76730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:32.636588 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954cc2d-5af0-4d50-89af-4cb8a8b76730 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:32.636874 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954cc2d-5af0-4d50-89af-4cb8a8b76730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:32.637155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954cc2d-5af0-4d50-89af-4cb8a8b76730 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:32.637850 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1954cc2d-5af0-4d50-89af-4cb8a8b76730 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:32.639820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1954cc2d-5af0-4d50-89af-4cb8a8b76730 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:32.769723 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1308/2627] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 15:03:32 2022] POST /identity/v3/projects => generated 401 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:03:32.775021 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fdf51-2789-40a6-95bc-7ab94ef541a4 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:32.780591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:03:32.786405 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fdf51-2789-40a6-95bc-7ab94ef541a4 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:32.857357 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4510a89-0bb9-4b94-be67-51044088f683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:32.858070 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4510a89-0bb9-4b94-be67-51044088f683 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:32.858355 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4510a89-0bb9-4b94-be67-51044088f683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:32.858635 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4510a89-0bb9-4b94-be67-51044088f683 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:32.859307 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4510a89-0bb9-4b94-be67-51044088f683 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:32.861538 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4510a89-0bb9-4b94-be67-51044088f683 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:32.963944 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1320/2628] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 15:03:32 2022] POST /identity/v3/users => generated 307 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:03:32.968077 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954cc2d-5af0-4d50-89af-4cb8a8b76730 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:32.979182 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954cc2d-5af0-4d50-89af-4cb8a8b76730 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:32.984827 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d6e1a-842f-4eba-9eee-c183c781144e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:32.985467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d6e1a-842f-4eba-9eee-c183c781144e admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:32.987162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d6e1a-842f-4eba-9eee-c183c781144e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:32.987162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d6e1a-842f-4eba-9eee-c183c781144e admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:32.987162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d6e1a-842f-4eba-9eee-c183c781144e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:32.988493 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d6e1a-842f-4eba-9eee-c183c781144e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:32.999317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1309/2629] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:03:33.005321 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4510a89-0bb9-4b94-be67-51044088f683 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:33.015838 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4510a89-0bb9-4b94-be67-51044088f683 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:33.021111 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8590d4b-3d86-498d-887e-542e5afdea33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:33.021811 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8590d4b-3d86-498d-887e-542e5afdea33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:33.022097 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8590d4b-3d86-498d-887e-542e5afdea33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:33.022644 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8590d4b-3d86-498d-887e-542e5afdea33 admin admin] PATH_INFO: `/v3/projects/2e39de17763c4744a50195994e17ea1e/users/2792faa1fef4456d9b74f209fd35e3a3/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:33.038854 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8590d4b-3d86-498d-887e-542e5afdea33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e39de17763c4744a50195994e17ea1e, user_id=2792faa1fef4456d9b74f209fd35e3a3, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:33.041704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8590d4b-3d86-498d-887e-542e5afdea33 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:33.165060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1321/2630] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:03:33 2022] PUT /identity/v3/projects/2e39de17763c4744a50195994e17ea1e/users/2792faa1fef4456d9b74f209fd35e3a3/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:03:33.168921 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d6e1a-842f-4eba-9eee-c183c781144e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:33.179430 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d6e1a-842f-4eba-9eee-c183c781144e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:33.239002 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4090d872-0c51-4c59-8dd4-ef6a0501e128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:33.239663 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090d872-0c51-4c59-8dd4-ef6a0501e128 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:33.239977 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090d872-0c51-4c59-8dd4-ef6a0501e128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:33.240256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090d872-0c51-4c59-8dd4-ef6a0501e128 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:33.240956 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4090d872-0c51-4c59-8dd4-ef6a0501e128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:33.242551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4090d872-0c51-4c59-8dd4-ef6a0501e128 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:33.252639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1310/2631] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:03:33.256841 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8590d4b-3d86-498d-887e-542e5afdea33 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:33.267386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8590d4b-3d86-498d-887e-542e5afdea33 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:33.272845 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2acc7d-19e5-4280-9ffb-a86d78ac15fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:33.273500 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2acc7d-19e5-4280-9ffb-a86d78ac15fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:33.273782 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2acc7d-19e5-4280-9ffb-a86d78ac15fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:33.274052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2acc7d-19e5-4280-9ffb-a86d78ac15fb admin admin] PATH_INFO: `/v3/projects/2e39de17763c4744a50195994e17ea1e/users/2792faa1fef4456d9b74f209fd35e3a3/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:33.276088 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2acc7d-19e5-4280-9ffb-a86d78ac15fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e39de17763c4744a50195994e17ea1e, user_id=2792faa1fef4456d9b74f209fd35e3a3, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:33.281201 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2acc7d-19e5-4280-9ffb-a86d78ac15fb admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:33.292568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1322/2632] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:03:33 2022] PUT /identity/v3/projects/2e39de17763c4744a50195994e17ea1e/users/2792faa1fef4456d9b74f209fd35e3a3/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:03:33.302991 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72274c2-f2b9-4a01-a602-9ae768d256be None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:33.303356 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72274c2-f2b9-4a01-a602-9ae768d256be None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:33.303622 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72274c2-f2b9-4a01-a602-9ae768d256be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:33.333483 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e72274c2-f2b9-4a01-a602-9ae768d256be None None] MFA Rules not processed for user `2792faa1fef4456d9b74f209fd35e3a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:03:33.334702 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e72274c2-f2b9-4a01-a602-9ae768d256be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:33.404945 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1311/2633] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:03:33 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:03:33.415728 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2acc7d-19e5-4280-9ffb-a86d78ac15fb admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:33.417373 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ad2acc7d-19e5-4280-9ffb-a86d78ac15fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:33.475177 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2acc7d-19e5-4280-9ffb-a86d78ac15fb admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:33.479479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5761053-ce8f-47d2-83ad-c0b2032cb6f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:33.480090 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5761053-ce8f-47d2-83ad-c0b2032cb6f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:33.480372 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5761053-ce8f-47d2-83ad-c0b2032cb6f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:33.480647 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5761053-ce8f-47d2-83ad-c0b2032cb6f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:33.489730 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5761053-ce8f-47d2-83ad-c0b2032cb6f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:33.491151 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5761053-ce8f-47d2-83ad-c0b2032cb6f5 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:33.505042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1323/2634] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:03:33 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:03:33.818145 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72274c2-f2b9-4a01-a602-9ae768d256be None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:33.827876 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72274c2-f2b9-4a01-a602-9ae768d256be None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:33.832129 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f7fb9-7661-443d-8dc0-2cb8ed424564 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:33.832470 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f7fb9-7661-443d-8dc0-2cb8ed424564 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:33.834428 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f7fb9-7661-443d-8dc0-2cb8ed424564 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:33.834428 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f7fb9-7661-443d-8dc0-2cb8ed424564 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:33.878479 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:03:33.891634 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f7fb9-7661-443d-8dc0-2cb8ed424564 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:33.893149 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f7fb9-7661-443d-8dc0-2cb8ed424564 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:33.902265 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1312/2635] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:03:33 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:03:41.040907 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5761053-ce8f-47d2-83ad-c0b2032cb6f5 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:41.101512 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:03:41.105586 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5761053-ce8f-47d2-83ad-c0b2032cb6f5 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:41.110130 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce344b-0bfd-4ad3-8d34-4872f95b2cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:41.110763 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce344b-0bfd-4ad3-8d34-4872f95b2cca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:41.111044 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce344b-0bfd-4ad3-8d34-4872f95b2cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:41.111322 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce344b-0bfd-4ad3-8d34-4872f95b2cca admin admin] PATH_INFO: `/v3/users/b6318b13ac6842b5b31d4e1afa2ade4b` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:41.112520 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce344b-0bfd-4ad3-8d34-4872f95b2cca admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6318b13ac6842b5b31d4e1afa2ade4b)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:41.114436 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce344b-0bfd-4ad3-8d34-4872f95b2cca admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:41.483926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f2ce344b-0bfd-4ad3-8d34-4872f95b2cca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6318b13ac6842b5b31d4e1afa2ade4b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:41.484584 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f2ce344b-0bfd-4ad3-8d34-4872f95b2cca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6318b13ac6842b5b31d4e1afa2ade4b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:41.496989 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f2ce344b-0bfd-4ad3-8d34-4872f95b2cca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6318b13ac6842b5b31d4e1afa2ade4b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:41.503690 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f2ce344b-0bfd-4ad3-8d34-4872f95b2cca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6318b13ac6842b5b31d4e1afa2ade4b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:41.504853 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f2ce344b-0bfd-4ad3-8d34-4872f95b2cca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6318b13ac6842b5b31d4e1afa2ade4b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:41.639897 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1324/2636] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:03:41 2022] DELETE /identity/v3/users/b6318b13ac6842b5b31d4e1afa2ade4b => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:03:41.644364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd32e5-b65d-4332-bb80-432ec066cdb9 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:41.644936 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd32e5-b65d-4332-bb80-432ec066cdb9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:41.644936 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd32e5-b65d-4332-bb80-432ec066cdb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:41.666914 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c8bd32e5-b65d-4332-bb80-432ec066cdb9 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:03:41.668418 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c8bd32e5-b65d-4332-bb80-432ec066cdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:41.727811 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1313/2637] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:03:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:03:41.738350 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce344b-0bfd-4ad3-8d34-4872f95b2cca admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:41.740361 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f2ce344b-0bfd-4ad3-8d34-4872f95b2cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:41.807677 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce344b-0bfd-4ad3-8d34-4872f95b2cca admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:41.811830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e3300-c773-48ad-90cd-c64aa511a415 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:41.812448 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e3300-c773-48ad-90cd-c64aa511a415 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:41.813144 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e3300-c773-48ad-90cd-c64aa511a415 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:41.813854 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e3300-c773-48ad-90cd-c64aa511a415 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:41.823427 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e3300-c773-48ad-90cd-c64aa511a415 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:41.824881 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e3300-c773-48ad-90cd-c64aa511a415 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:41.835516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1325/2638] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:03:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:03:42.346546 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd32e5-b65d-4332-bb80-432ec066cdb9 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:42.348589 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c8bd32e5-b65d-4332-bb80-432ec066cdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:42.360993 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd32e5-b65d-4332-bb80-432ec066cdb9 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:42.366924 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b0e611-0bb9-4a24-9811-52f12594b8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:42.370528 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0e611-0bb9-4a24-9811-52f12594b8d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:42.370528 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0e611-0bb9-4a24-9811-52f12594b8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:42.370528 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0e611-0bb9-4a24-9811-52f12594b8d8 admin admin] PATH_INFO: `/v3/projects/be3967716e0e45c3b691af0e01e04d0c` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:42.370528 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b0e611-0bb9-4a24-9811-52f12594b8d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be3967716e0e45c3b691af0e01e04d0c)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:42.371355 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b0e611-0bb9-4a24-9811-52f12594b8d8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:42.502791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-46b0e611-0bb9-4a24-9811-52f12594b8d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be3967716e0e45c3b691af0e01e04d0c', 'request_id': 'req-46b0e611-0bb9-4a24-9811-52f12594b8d8'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:42.508250 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-46b0e611-0bb9-4a24-9811-52f12594b8d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be3967716e0e45c3b691af0e01e04d0c', 'request_id': 'req-46b0e611-0bb9-4a24-9811-52f12594b8d8'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:42.573112 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-46b0e611-0bb9-4a24-9811-52f12594b8d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be3967716e0e45c3b691af0e01e04d0c', 'request_id': 'req-46b0e611-0bb9-4a24-9811-52f12594b8d8'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:42.584758 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-46b0e611-0bb9-4a24-9811-52f12594b8d8 admin admin] The token cache is being invalidate because project be3967716e0e45c3b691af0e01e04d0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:03:42.585045 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-46b0e611-0bb9-4a24-9811-52f12594b8d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:42.585688 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-46b0e611-0bb9-4a24-9811-52f12594b8d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:42.587224 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1314/2639] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:03:42 2022] DELETE /identity/v3/projects/be3967716e0e45c3b691af0e01e04d0c => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:03:42.591295 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e3300-c773-48ad-90cd-c64aa511a415 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:42.601812 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e3300-c773-48ad-90cd-c64aa511a415 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:42.659872 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee4a708-e11b-415f-a8f4-f91d146c3abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:42.661031 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee4a708-e11b-415f-a8f4-f91d146c3abc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:42.661317 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee4a708-e11b-415f-a8f4-f91d146c3abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:42.661595 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee4a708-e11b-415f-a8f4-f91d146c3abc admin admin] PATH_INFO: `/v3/users/2792faa1fef4456d9b74f209fd35e3a3` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:42.662784 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee4a708-e11b-415f-a8f4-f91d146c3abc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2792faa1fef4456d9b74f209fd35e3a3)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:42.664807 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee4a708-e11b-415f-a8f4-f91d146c3abc admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:42.687273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:03:42.961708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cee4a708-e11b-415f-a8f4-f91d146c3abc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2792faa1fef4456d9b74f209fd35e3a3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:42.964018 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cee4a708-e11b-415f-a8f4-f91d146c3abc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2792faa1fef4456d9b74f209fd35e3a3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:42.980582 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cee4a708-e11b-415f-a8f4-f91d146c3abc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2792faa1fef4456d9b74f209fd35e3a3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:42.990091 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cee4a708-e11b-415f-a8f4-f91d146c3abc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2792faa1fef4456d9b74f209fd35e3a3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:42.991073 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cee4a708-e11b-415f-a8f4-f91d146c3abc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2792faa1fef4456d9b74f209fd35e3a3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:43.122265 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1326/2640] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:03:42 2022] DELETE /identity/v3/users/2792faa1fef4456d9b74f209fd35e3a3 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:03:43.643140 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b0e611-0bb9-4a24-9811-52f12594b8d8 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:43.644638 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-46b0e611-0bb9-4a24-9811-52f12594b8d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:43.658481 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b0e611-0bb9-4a24-9811-52f12594b8d8 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:43.722067 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8a32f-d70f-4aec-bd86-a24107e2f1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:43.722792 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8a32f-d70f-4aec-bd86-a24107e2f1fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:43.723116 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8a32f-d70f-4aec-bd86-a24107e2f1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:43.723393 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8a32f-d70f-4aec-bd86-a24107e2f1fc admin admin] PATH_INFO: `/v3/projects/2e39de17763c4744a50195994e17ea1e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:43.724683 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d8a32f-d70f-4aec-bd86-a24107e2f1fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e39de17763c4744a50195994e17ea1e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:43.726583 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d8a32f-d70f-4aec-bd86-a24107e2f1fc admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:43.737471 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:03:43.853335 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-90d8a32f-d70f-4aec-bd86-a24107e2f1fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e39de17763c4744a50195994e17ea1e', 'request_id': 'req-90d8a32f-d70f-4aec-bd86-a24107e2f1fc'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:43.859199 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-90d8a32f-d70f-4aec-bd86-a24107e2f1fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e39de17763c4744a50195994e17ea1e', 'request_id': 'req-90d8a32f-d70f-4aec-bd86-a24107e2f1fc'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:43.971626 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-90d8a32f-d70f-4aec-bd86-a24107e2f1fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e39de17763c4744a50195994e17ea1e', 'request_id': 'req-90d8a32f-d70f-4aec-bd86-a24107e2f1fc'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:43.984005 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-90d8a32f-d70f-4aec-bd86-a24107e2f1fc admin admin] The token cache is being invalidate because project 2e39de17763c4744a50195994e17ea1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:03:43.984299 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-90d8a32f-d70f-4aec-bd86-a24107e2f1fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:43.987270 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-90d8a32f-d70f-4aec-bd86-a24107e2f1fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:43.987270 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1315/2641] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:03:43 2022] DELETE /identity/v3/projects/2e39de17763c4744a50195994e17ea1e => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:03:43.992144 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13804621-086b-4725-9790-af592e32dbc1 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:43.992630 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13804621-086b-4725-9790-af592e32dbc1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:43.992738 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13804621-086b-4725-9790-af592e32dbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:44.013301 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-13804621-086b-4725-9790-af592e32dbc1 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:03:44.014506 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-13804621-086b-4725-9790-af592e32dbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:44.077496 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1327/2642] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:03:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:03:44.086513 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488f492-a038-4efe-9dde-d6bae665320b None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:44.086805 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488f492-a038-4efe-9dde-d6bae665320b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:44.087115 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488f492-a038-4efe-9dde-d6bae665320b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:44.107032 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-d488f492-a038-4efe-9dde-d6bae665320b None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:03:44.108346 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d488f492-a038-4efe-9dde-d6bae665320b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:44.116718 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1316/2643] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:03:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:03:44.121249 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13804621-086b-4725-9790-af592e32dbc1 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:44.131534 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13804621-086b-4725-9790-af592e32dbc1 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:44.136133 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2a34da-5aa1-46bd-9291-6e76c20a2187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:44.136744 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a34da-5aa1-46bd-9291-6e76c20a2187 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:44.137030 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a34da-5aa1-46bd-9291-6e76c20a2187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:44.137305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a34da-5aa1-46bd-9291-6e76c20a2187 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:44.137997 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2a34da-5aa1-46bd-9291-6e76c20a2187 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:03:44.138299 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2a34da-5aa1-46bd-9291-6e76c20a2187 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:44.139910 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2a34da-5aa1-46bd-9291-6e76c20a2187 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:44.152358 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1328/2644] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 15:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:03:44.156895 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488f492-a038-4efe-9dde-d6bae665320b None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:44.166174 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488f492-a038-4efe-9dde-d6bae665320b None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:44.170895 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f955903-24d9-4d14-8e86-06128261e7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:44.171539 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f955903-24d9-4d14-8e86-06128261e7f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:44.171822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f955903-24d9-4d14-8e86-06128261e7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:44.172099 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f955903-24d9-4d14-8e86-06128261e7f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:44.172768 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f955903-24d9-4d14-8e86-06128261e7f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:44.174705 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f955903-24d9-4d14-8e86-06128261e7f5 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:44.296016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1317/2645] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 15:03:44 2022] POST /identity/v3/projects => generated 379 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:03:44.299784 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2a34da-5aa1-46bd-9291-6e76c20a2187 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:44.310300 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2a34da-5aa1-46bd-9291-6e76c20a2187 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:44.365746 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1513a69b-f977-4d14-a128-fd2d00c52856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:44.366484 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513a69b-f977-4d14-a128-fd2d00c52856 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:44.366796 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513a69b-f977-4d14-a128-fd2d00c52856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:44.367060 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513a69b-f977-4d14-a128-fd2d00c52856 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:44.367771 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1513a69b-f977-4d14-a128-fd2d00c52856 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:44.369700 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1513a69b-f977-4d14-a128-fd2d00c52856 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:44.473575 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1329/2646] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 15:03:44 2022] POST /identity/v3/users => generated 290 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:03:44.479351 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f955903-24d9-4d14-8e86-06128261e7f5 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:44.490431 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f955903-24d9-4d14-8e86-06128261e7f5 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:44.496364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b53a06-89dc-4603-a547-517240b1f2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:44.497483 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b53a06-89dc-4603-a547-517240b1f2df admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:44.497876 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b53a06-89dc-4603-a547-517240b1f2df admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:44.498698 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b53a06-89dc-4603-a547-517240b1f2df admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:44.499072 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b53a06-89dc-4603-a547-517240b1f2df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:44.500995 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b53a06-89dc-4603-a547-517240b1f2df admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:44.516542 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1318/2647] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:03:44.526791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1513a69b-f977-4d14-a128-fd2d00c52856 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:44.539512 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1513a69b-f977-4d14-a128-fd2d00c52856 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:44.544238 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9d8fa-d7d9-4d99-95a7-5d0018a139b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:44.544867 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9d8fa-d7d9-4d99-95a7-5d0018a139b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:44.545149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9d8fa-d7d9-4d99-95a7-5d0018a139b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:44.545430 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9d8fa-d7d9-4d99-95a7-5d0018a139b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:44.546119 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f9d8fa-d7d9-4d99-95a7-5d0018a139b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:44.547808 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f9d8fa-d7d9-4d99-95a7-5d0018a139b4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:44.559609 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1330/2648] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:03:44.567475 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b53a06-89dc-4603-a547-517240b1f2df admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:44.577721 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b53a06-89dc-4603-a547-517240b1f2df admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:44.582655 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e792f8d-023c-4350-aab0-c21ad2cc4669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:44.583342 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e792f8d-023c-4350-aab0-c21ad2cc4669 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:44.583663 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e792f8d-023c-4350-aab0-c21ad2cc4669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:44.583945 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e792f8d-023c-4350-aab0-c21ad2cc4669 admin admin] PATH_INFO: `/v3/projects/680bc49225cd494792900474b0ad1402/users/334c6ea61b4b4fec8b6178ca9a500fb1/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:44.599882 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e792f8d-023c-4350-aab0-c21ad2cc4669 admin admin] RBAC: Authorizing `identity:create_grant(project_id=680bc49225cd494792900474b0ad1402, user_id=334c6ea61b4b4fec8b6178ca9a500fb1, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:44.602674 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e792f8d-023c-4350-aab0-c21ad2cc4669 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:44.733963 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1319/2649] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:03:44 2022] PUT /identity/v3/projects/680bc49225cd494792900474b0ad1402/users/334c6ea61b4b4fec8b6178ca9a500fb1/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:03:44.744614 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7ecec-0277-4eca-9118-b4e12598594c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:44.744905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7ecec-0277-4eca-9118-b4e12598594c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:44.745217 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7ecec-0277-4eca-9118-b4e12598594c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:44.753443 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:03:44.774340 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0bd7ecec-0277-4eca-9118-b4e12598594c None None] MFA Rules not processed for user `334c6ea61b4b4fec8b6178ca9a500fb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:03:44.775588 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0bd7ecec-0277-4eca-9118-b4e12598594c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:44.839024 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1331/2650] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:03:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:03:44.843627 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e792f8d-023c-4350-aab0-c21ad2cc4669 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:44.905930 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e792f8d-023c-4350-aab0-c21ad2cc4669 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:44.912170 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c1d56-1b61-41ff-a715-b068535dcd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:44.912833 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c1d56-1b61-41ff-a715-b068535dcd23 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:44.913118 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c1d56-1b61-41ff-a715-b068535dcd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:44.913387 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c1d56-1b61-41ff-a715-b068535dcd23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:44.914050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471c1d56-1b61-41ff-a715-b068535dcd23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:44.916096 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471c1d56-1b61-41ff-a715-b068535dcd23 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:45.029407 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1320/2651] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 15:03:44 2022] POST /identity/v3/projects => generated 379 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:03:45.030953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7ecec-0277-4eca-9118-b4e12598594c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:45.041865 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7ecec-0277-4eca-9118-b4e12598594c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:45.105116 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a27065-fc8d-4bc2-816d-392a09f585e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:45.105737 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a27065-fc8d-4bc2-816d-392a09f585e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:45.106026 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a27065-fc8d-4bc2-816d-392a09f585e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:45.106305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a27065-fc8d-4bc2-816d-392a09f585e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:45.107075 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a27065-fc8d-4bc2-816d-392a09f585e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:45.109128 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a27065-fc8d-4bc2-816d-392a09f585e1 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:45.219855 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1332/2652] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 15:03:45 2022] POST /identity/v3/users => generated 296 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:03:45.223574 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c1d56-1b61-41ff-a715-b068535dcd23 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:45.233918 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c1d56-1b61-41ff-a715-b068535dcd23 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:45.239054 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a3766-b92e-4a32-91b7-3b8cde137b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:45.239701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a3766-b92e-4a32-91b7-3b8cde137b9a admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:45.239983 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a3766-b92e-4a32-91b7-3b8cde137b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:45.240372 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a3766-b92e-4a32-91b7-3b8cde137b9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:45.242884 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a3766-b92e-4a32-91b7-3b8cde137b9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:45.242884 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a3766-b92e-4a32-91b7-3b8cde137b9a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:45.259909 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1321/2653] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:03:45.264268 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a27065-fc8d-4bc2-816d-392a09f585e1 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:45.274799 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a27065-fc8d-4bc2-816d-392a09f585e1 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:45.279354 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebaa56-3cd5-4928-a5e8-be1913c4613a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:45.280048 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebaa56-3cd5-4928-a5e8-be1913c4613a admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:45.280336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebaa56-3cd5-4928-a5e8-be1913c4613a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:45.280617 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebaa56-3cd5-4928-a5e8-be1913c4613a admin admin] PATH_INFO: `/v3/projects/08ba101bfa48487ab6616e21b959beca/users/8cb8fa1da1bf45869219944968b62305/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:45.295516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ebaa56-3cd5-4928-a5e8-be1913c4613a admin admin] RBAC: Authorizing `identity:create_grant(project_id=08ba101bfa48487ab6616e21b959beca, user_id=8cb8fa1da1bf45869219944968b62305, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:45.298185 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ebaa56-3cd5-4928-a5e8-be1913c4613a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:45.391202 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1333/2654] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:03:45 2022] PUT /identity/v3/projects/08ba101bfa48487ab6616e21b959beca/users/8cb8fa1da1bf45869219944968b62305/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:03:45.395456 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a3766-b92e-4a32-91b7-3b8cde137b9a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:45.405541 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a3766-b92e-4a32-91b7-3b8cde137b9a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:45.410558 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:03:45.462596 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af87e4-2472-467f-bd12-8233d248192b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:45.463213 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af87e4-2472-467f-bd12-8233d248192b admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:45.467256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af87e4-2472-467f-bd12-8233d248192b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:45.467256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af87e4-2472-467f-bd12-8233d248192b admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:45.467256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af87e4-2472-467f-bd12-8233d248192b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:45.467256 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af87e4-2472-467f-bd12-8233d248192b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:45.476834 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1322/2655] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:03:45.481218 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebaa56-3cd5-4928-a5e8-be1913c4613a admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:45.492009 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebaa56-3cd5-4928-a5e8-be1913c4613a admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:45.497024 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a21a181-2aff-4214-b7ff-882bff8ab01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:45.503734 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21a181-2aff-4214-b7ff-882bff8ab01b admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:45.503734 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21a181-2aff-4214-b7ff-882bff8ab01b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:45.503734 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21a181-2aff-4214-b7ff-882bff8ab01b admin admin] PATH_INFO: `/v3/projects/08ba101bfa48487ab6616e21b959beca/users/8cb8fa1da1bf45869219944968b62305/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:45.503734 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a21a181-2aff-4214-b7ff-882bff8ab01b admin admin] RBAC: Authorizing `identity:create_grant(project_id=08ba101bfa48487ab6616e21b959beca, user_id=8cb8fa1da1bf45869219944968b62305, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:45.503734 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a21a181-2aff-4214-b7ff-882bff8ab01b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:45.513061 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1334/2656] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:03:45 2022] PUT /identity/v3/projects/08ba101bfa48487ab6616e21b959beca/users/8cb8fa1da1bf45869219944968b62305/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:03:45.523514 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d755d-be0e-436c-aa5b-efd77dc51d68 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:45.523804 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d755d-be0e-436c-aa5b-efd77dc51d68 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:45.524081 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d755d-be0e-436c-aa5b-efd77dc51d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:45.553689 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b35d755d-be0e-436c-aa5b-efd77dc51d68 None None] MFA Rules not processed for user `8cb8fa1da1bf45869219944968b62305`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:03:45.554942 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b35d755d-be0e-436c-aa5b-efd77dc51d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:45.755998 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1323/2657] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:03:45 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:03:45.767822 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a21a181-2aff-4214-b7ff-882bff8ab01b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:45.769599 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4a21a181-2aff-4214-b7ff-882bff8ab01b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:45.828975 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:03:45.834233 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a21a181-2aff-4214-b7ff-882bff8ab01b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:45.837845 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a4a25-bff5-4c49-acd1-3139a7112020 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:45.838451 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a4a25-bff5-4c49-acd1-3139a7112020 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:45.838729 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a4a25-bff5-4c49-acd1-3139a7112020 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:45.838999 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a4a25-bff5-4c49-acd1-3139a7112020 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:45.848032 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16a4a25-bff5-4c49-acd1-3139a7112020 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:45.849469 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16a4a25-bff5-4c49-acd1-3139a7112020 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:45.859100 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1335/2658] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:03:45 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:03:45.964428 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d755d-be0e-436c-aa5b-efd77dc51d68 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:45.974328 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d755d-be0e-436c-aa5b-efd77dc51d68 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:45.980029 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75fcd15-993f-4312-803d-305222c17733 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:45.981381 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fcd15-993f-4312-803d-305222c17733 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:45.981381 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fcd15-993f-4312-803d-305222c17733 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:45.981381 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fcd15-993f-4312-803d-305222c17733 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:46.052417 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75fcd15-993f-4312-803d-305222c17733 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:46.057677 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75fcd15-993f-4312-803d-305222c17733 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:46.067436 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1324/2659] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:03:45 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:03:50.393158 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a4a25-bff5-4c49-acd1-3139a7112020 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:50.465171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a4a25-bff5-4c49-acd1-3139a7112020 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:50.470260 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a93dd-0e90-49ba-8508-6dff5d420ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:50.473385 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a93dd-0e90-49ba-8508-6dff5d420ba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:50.473385 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a93dd-0e90-49ba-8508-6dff5d420ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:50.473385 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a93dd-0e90-49ba-8508-6dff5d420ba0 admin admin] PATH_INFO: `/v3/users/334c6ea61b4b4fec8b6178ca9a500fb1` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:50.473385 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a93dd-0e90-49ba-8508-6dff5d420ba0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=334c6ea61b4b4fec8b6178ca9a500fb1)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:50.474887 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a93dd-0e90-49ba-8508-6dff5d420ba0 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:50.642033 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e44a93dd-0e90-49ba-8508-6dff5d420ba0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '334c6ea61b4b4fec8b6178ca9a500fb1'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:50.642624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e44a93dd-0e90-49ba-8508-6dff5d420ba0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '334c6ea61b4b4fec8b6178ca9a500fb1'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:50.654882 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e44a93dd-0e90-49ba-8508-6dff5d420ba0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '334c6ea61b4b4fec8b6178ca9a500fb1'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:50.662422 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e44a93dd-0e90-49ba-8508-6dff5d420ba0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '334c6ea61b4b4fec8b6178ca9a500fb1'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:50.663605 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e44a93dd-0e90-49ba-8508-6dff5d420ba0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '334c6ea61b4b4fec8b6178ca9a500fb1'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:50.767145 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1336/2660] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:03:50 2022] DELETE /identity/v3/users/334c6ea61b4b4fec8b6178ca9a500fb1 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:03:50.772640 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59657cc-e6f7-4fe9-9ee4-b368ba64d6e7 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:50.773147 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59657cc-e6f7-4fe9-9ee4-b368ba64d6e7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:50.773248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59657cc-e6f7-4fe9-9ee4-b368ba64d6e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:50.791447 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b59657cc-e6f7-4fe9-9ee4-b368ba64d6e7 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:03:50.792676 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b59657cc-e6f7-4fe9-9ee4-b368ba64d6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:50.818262 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:03:50.853796 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1325/2661] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:03:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:03:50.863660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a93dd-0e90-49ba-8508-6dff5d420ba0 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:50.865225 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e44a93dd-0e90-49ba-8508-6dff5d420ba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:50.931125 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a93dd-0e90-49ba-8508-6dff5d420ba0 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:50.936246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ca137-47df-44e0-9c66-df215287d819 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:50.936246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ca137-47df-44e0-9c66-df215287d819 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:50.936246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ca137-47df-44e0-9c66-df215287d819 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:50.937275 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ca137-47df-44e0-9c66-df215287d819 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:50.946571 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3ca137-47df-44e0-9c66-df215287d819 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:50.948369 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3ca137-47df-44e0-9c66-df215287d819 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:50.958898 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1337/2662] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:03:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:03:51.454763 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59657cc-e6f7-4fe9-9ee4-b368ba64d6e7 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:51.456223 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b59657cc-e6f7-4fe9-9ee4-b368ba64d6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:51.469266 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59657cc-e6f7-4fe9-9ee4-b368ba64d6e7 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:51.474055 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b63eb5-b59b-4d7b-8ec2-2b20f6e81326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:51.474704 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b63eb5-b59b-4d7b-8ec2-2b20f6e81326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:51.474989 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b63eb5-b59b-4d7b-8ec2-2b20f6e81326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:51.475268 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b63eb5-b59b-4d7b-8ec2-2b20f6e81326 admin admin] PATH_INFO: `/v3/projects/680bc49225cd494792900474b0ad1402` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:51.476478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b63eb5-b59b-4d7b-8ec2-2b20f6e81326 admin admin] RBAC: Authorizing `identity:delete_project(project_id=680bc49225cd494792900474b0ad1402)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:51.478474 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b63eb5-b59b-4d7b-8ec2-2b20f6e81326 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:51.609011 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c3b63eb5-b59b-4d7b-8ec2-2b20f6e81326 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '680bc49225cd494792900474b0ad1402', 'request_id': 'req-c3b63eb5-b59b-4d7b-8ec2-2b20f6e81326'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:51.615319 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c3b63eb5-b59b-4d7b-8ec2-2b20f6e81326 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '680bc49225cd494792900474b0ad1402', 'request_id': 'req-c3b63eb5-b59b-4d7b-8ec2-2b20f6e81326'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:51.693207 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c3b63eb5-b59b-4d7b-8ec2-2b20f6e81326 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '680bc49225cd494792900474b0ad1402', 'request_id': 'req-c3b63eb5-b59b-4d7b-8ec2-2b20f6e81326'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:51.708020 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c3b63eb5-b59b-4d7b-8ec2-2b20f6e81326 admin admin] The token cache is being invalidate because project 680bc49225cd494792900474b0ad1402 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:03:51.708348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c3b63eb5-b59b-4d7b-8ec2-2b20f6e81326 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:51.709086 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c3b63eb5-b59b-4d7b-8ec2-2b20f6e81326 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:51.710944 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1326/2663] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:03:51 2022] DELETE /identity/v3/projects/680bc49225cd494792900474b0ad1402 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:03:51.715063 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ca137-47df-44e0-9c66-df215287d819 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:51.725214 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ca137-47df-44e0-9c66-df215287d819 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:51.732899 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:03:51.786137 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e831d-a3c8-4070-8859-654f8b8db1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:51.786769 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e831d-a3c8-4070-8859-654f8b8db1b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:51.787053 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e831d-a3c8-4070-8859-654f8b8db1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:51.787333 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e831d-a3c8-4070-8859-654f8b8db1b5 admin admin] PATH_INFO: `/v3/users/8cb8fa1da1bf45869219944968b62305` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:51.788534 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41e831d-a3c8-4070-8859-654f8b8db1b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cb8fa1da1bf45869219944968b62305)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:51.790751 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41e831d-a3c8-4070-8859-654f8b8db1b5 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:52.031785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c41e831d-a3c8-4070-8859-654f8b8db1b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cb8fa1da1bf45869219944968b62305'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:52.032454 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c41e831d-a3c8-4070-8859-654f8b8db1b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cb8fa1da1bf45869219944968b62305'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:52.046141 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c41e831d-a3c8-4070-8859-654f8b8db1b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cb8fa1da1bf45869219944968b62305'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:52.053906 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c41e831d-a3c8-4070-8859-654f8b8db1b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cb8fa1da1bf45869219944968b62305'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:52.054658 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c41e831d-a3c8-4070-8859-654f8b8db1b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cb8fa1da1bf45869219944968b62305'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:52.126427 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1338/2664] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:03:51 2022] DELETE /identity/v3/users/8cb8fa1da1bf45869219944968b62305 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:03:52.728984 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b63eb5-b59b-4d7b-8ec2-2b20f6e81326 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:52.730484 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c3b63eb5-b59b-4d7b-8ec2-2b20f6e81326 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:52.744138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b63eb5-b59b-4d7b-8ec2-2b20f6e81326 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:52.802456 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9dff9a-9b5e-44a5-a208-7b4cb3fa5f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:52.803096 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9dff9a-9b5e-44a5-a208-7b4cb3fa5f6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:52.803449 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9dff9a-9b5e-44a5-a208-7b4cb3fa5f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:52.803698 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9dff9a-9b5e-44a5-a208-7b4cb3fa5f6c admin admin] PATH_INFO: `/v3/projects/08ba101bfa48487ab6616e21b959beca` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:52.804949 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9dff9a-9b5e-44a5-a208-7b4cb3fa5f6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=08ba101bfa48487ab6616e21b959beca)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:52.806848 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9dff9a-9b5e-44a5-a208-7b4cb3fa5f6c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:52.930350 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-4c9dff9a-9b5e-44a5-a208-7b4cb3fa5f6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08ba101bfa48487ab6616e21b959beca', 'request_id': 'req-4c9dff9a-9b5e-44a5-a208-7b4cb3fa5f6c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:52.934329 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:03:52.937290 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-4c9dff9a-9b5e-44a5-a208-7b4cb3fa5f6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08ba101bfa48487ab6616e21b959beca', 'request_id': 'req-4c9dff9a-9b5e-44a5-a208-7b4cb3fa5f6c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:53.023123 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-4c9dff9a-9b5e-44a5-a208-7b4cb3fa5f6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08ba101bfa48487ab6616e21b959beca', 'request_id': 'req-4c9dff9a-9b5e-44a5-a208-7b4cb3fa5f6c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:53.036641 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-4c9dff9a-9b5e-44a5-a208-7b4cb3fa5f6c admin admin] The token cache is being invalidate because project 08ba101bfa48487ab6616e21b959beca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:03:53.036940 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-4c9dff9a-9b5e-44a5-a208-7b4cb3fa5f6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:53.037622 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-4c9dff9a-9b5e-44a5-a208-7b4cb3fa5f6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:53.039503 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1327/2665] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:03:52 2022] DELETE /identity/v3/projects/08ba101bfa48487ab6616e21b959beca => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:03:53.044550 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5745c8-b654-450e-b131-01b6959d2f43 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:53.045118 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5745c8-b654-450e-b131-01b6959d2f43 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:53.045118 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5745c8-b654-450e-b131-01b6959d2f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:53.065467 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0d5745c8-b654-450e-b131-01b6959d2f43 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:03:53.067270 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0d5745c8-b654-450e-b131-01b6959d2f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:53.130839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1339/2666] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:03:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:03:53.139621 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399bd863-c063-40ac-95c9-87359fda226c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:53.139914 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399bd863-c063-40ac-95c9-87359fda226c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:53.140191 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399bd863-c063-40ac-95c9-87359fda226c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:53.161344 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-399bd863-c063-40ac-95c9-87359fda226c None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:03:53.162584 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-399bd863-c063-40ac-95c9-87359fda226c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:53.170794 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1328/2667] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:03:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:03:53.174919 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5745c8-b654-450e-b131-01b6959d2f43 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:53.184728 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5745c8-b654-450e-b131-01b6959d2f43 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:53.189380 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30919a16-f23b-4b70-bf57-a60f4bd9ac9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:53.190000 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30919a16-f23b-4b70-bf57-a60f4bd9ac9b admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:53.190285 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30919a16-f23b-4b70-bf57-a60f4bd9ac9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:53.190562 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30919a16-f23b-4b70-bf57-a60f4bd9ac9b admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:53.191843 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30919a16-f23b-4b70-bf57-a60f4bd9ac9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:03:53.192224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30919a16-f23b-4b70-bf57-a60f4bd9ac9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:53.193819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30919a16-f23b-4b70-bf57-a60f4bd9ac9b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:53.206029 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1340/2668] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 15:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:03:53.212371 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399bd863-c063-40ac-95c9-87359fda226c None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:53.224139 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399bd863-c063-40ac-95c9-87359fda226c None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:53.228409 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac766f7-c30f-412c-b6df-8d3810f4d277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:53.229083 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac766f7-c30f-412c-b6df-8d3810f4d277 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:53.229366 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac766f7-c30f-412c-b6df-8d3810f4d277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:53.229649 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac766f7-c30f-412c-b6df-8d3810f4d277 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:53.230332 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac766f7-c30f-412c-b6df-8d3810f4d277 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:53.232693 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac766f7-c30f-412c-b6df-8d3810f4d277 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:53.416072 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1329/2669] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 15:03:53 2022] POST /identity/v3/projects => generated 361 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:03:53.418781 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30919a16-f23b-4b70-bf57-a60f4bd9ac9b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:53.435388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30919a16-f23b-4b70-bf57-a60f4bd9ac9b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:53.453881 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:03:53.503594 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed3d02-3401-48f9-bfc0-e9045457dd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:53.504231 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed3d02-3401-48f9-bfc0-e9045457dd3d admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:53.504512 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed3d02-3401-48f9-bfc0-e9045457dd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:53.504791 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed3d02-3401-48f9-bfc0-e9045457dd3d admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:53.505457 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed3d02-3401-48f9-bfc0-e9045457dd3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:53.507617 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed3d02-3401-48f9-bfc0-e9045457dd3d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:53.637259 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1341/2670] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 15:03:53 2022] POST /identity/v3/users => generated 281 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:03:53.641345 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac766f7-c30f-412c-b6df-8d3810f4d277 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:53.652172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac766f7-c30f-412c-b6df-8d3810f4d277 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:53.656798 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c7cb3-9bb9-4983-9bfd-a54213aa7ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:53.657462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c7cb3-9bb9-4983-9bfd-a54213aa7ee0 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:53.657748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c7cb3-9bb9-4983-9bfd-a54213aa7ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:53.658027 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c7cb3-9bb9-4983-9bfd-a54213aa7ee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:53.658849 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c7cb3-9bb9-4983-9bfd-a54213aa7ee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:53.660869 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c7cb3-9bb9-4983-9bfd-a54213aa7ee0 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:53.671214 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1330/2671] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:03:53.676228 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed3d02-3401-48f9-bfc0-e9045457dd3d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:53.686514 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed3d02-3401-48f9-bfc0-e9045457dd3d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:53.691694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0683714e-9b37-4de3-8060-01b870ed7f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:53.692303 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683714e-9b37-4de3-8060-01b870ed7f06 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:53.692645 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683714e-9b37-4de3-8060-01b870ed7f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:53.692863 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683714e-9b37-4de3-8060-01b870ed7f06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:53.693583 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0683714e-9b37-4de3-8060-01b870ed7f06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:53.695242 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0683714e-9b37-4de3-8060-01b870ed7f06 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:53.707785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1342/2672] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:03:53.718997 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c7cb3-9bb9-4983-9bfd-a54213aa7ee0 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:53.729123 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c7cb3-9bb9-4983-9bfd-a54213aa7ee0 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:53.733671 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a300cc-aa41-4f14-81db-82cb1d77e2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:53.734330 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a300cc-aa41-4f14-81db-82cb1d77e2b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:53.734616 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a300cc-aa41-4f14-81db-82cb1d77e2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:53.734896 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a300cc-aa41-4f14-81db-82cb1d77e2b7 admin admin] PATH_INFO: `/v3/projects/cea308312ea240a0be12f37e10a93b33/users/0291177cf60f41f9b27d88b36c46a2e0/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:53.749979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a300cc-aa41-4f14-81db-82cb1d77e2b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cea308312ea240a0be12f37e10a93b33, user_id=0291177cf60f41f9b27d88b36c46a2e0, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:53.752778 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a300cc-aa41-4f14-81db-82cb1d77e2b7 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:53.877295 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1331/2673] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:03:53 2022] PUT /identity/v3/projects/cea308312ea240a0be12f37e10a93b33/users/0291177cf60f41f9b27d88b36c46a2e0/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:03:53.889345 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d903f1c-23ec-4fbe-a0a7-9cdceb850d0f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:53.889587 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d903f1c-23ec-4fbe-a0a7-9cdceb850d0f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:53.889799 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d903f1c-23ec-4fbe-a0a7-9cdceb850d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:53.920083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-2d903f1c-23ec-4fbe-a0a7-9cdceb850d0f None None] MFA Rules not processed for user `0291177cf60f41f9b27d88b36c46a2e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:03:53.921354 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2d903f1c-23ec-4fbe-a0a7-9cdceb850d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:53.998516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1343/2674] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:03:53 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:03:54.040473 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a300cc-aa41-4f14-81db-82cb1d77e2b7 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:54.043225 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-77a300cc-aa41-4f14-81db-82cb1d77e2b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:54.115683 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a300cc-aa41-4f14-81db-82cb1d77e2b7 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:54.120597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7624af-03cc-4db4-ae72-09e5f60910f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:54.121228 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7624af-03cc-4db4-ae72-09e5f60910f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:54.121509 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7624af-03cc-4db4-ae72-09e5f60910f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:54.121851 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7624af-03cc-4db4-ae72-09e5f60910f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:54.132998 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7624af-03cc-4db4-ae72-09e5f60910f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:54.134422 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7624af-03cc-4db4-ae72-09e5f60910f6 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:54.144322 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1332/2675] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:03:54 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:03:54.162002 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d903f1c-23ec-4fbe-a0a7-9cdceb850d0f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:54.231723 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d903f1c-23ec-4fbe-a0a7-9cdceb850d0f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:54.236500 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cbaab-481f-4b90-80c1-fff4a4908ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:54.237150 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cbaab-481f-4b90-80c1-fff4a4908ac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:54.237428 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cbaab-481f-4b90-80c1-fff4a4908ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:54.238273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cbaab-481f-4b90-80c1-fff4a4908ac9 admin admin] PATH_INFO: `/v3/users/0291177cf60f41f9b27d88b36c46a2e0` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:54.239536 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18cbaab-481f-4b90-80c1-fff4a4908ac9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0291177cf60f41f9b27d88b36c46a2e0)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:54.241505 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18cbaab-481f-4b90-80c1-fff4a4908ac9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:54.696716 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f18cbaab-481f-4b90-80c1-fff4a4908ac9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0291177cf60f41f9b27d88b36c46a2e0'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:54.699769 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f18cbaab-481f-4b90-80c1-fff4a4908ac9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0291177cf60f41f9b27d88b36c46a2e0'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:54.703827 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:03:54.716673 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f18cbaab-481f-4b90-80c1-fff4a4908ac9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0291177cf60f41f9b27d88b36c46a2e0'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:54.726837 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f18cbaab-481f-4b90-80c1-fff4a4908ac9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0291177cf60f41f9b27d88b36c46a2e0'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:54.727621 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f18cbaab-481f-4b90-80c1-fff4a4908ac9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0291177cf60f41f9b27d88b36c46a2e0'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:54.852240 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1344/2676] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:03:54 2022] DELETE /identity/v3/users/0291177cf60f41f9b27d88b36c46a2e0 => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:03:54.855226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6453ce-6618-4831-89ac-dff3ef7487ed None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:54.856014 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6453ce-6618-4831-89ac-dff3ef7487ed None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:54.856439 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6453ce-6618-4831-89ac-dff3ef7487ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:54.879943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-1f6453ce-6618-4831-89ac-dff3ef7487ed None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:03:54.881492 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1f6453ce-6618-4831-89ac-dff3ef7487ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:54.970277 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1333/2677] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:03:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:03:54.986021 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cbaab-481f-4b90-80c1-fff4a4908ac9 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:54.987886 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f18cbaab-481f-4b90-80c1-fff4a4908ac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:55.064684 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cbaab-481f-4b90-80c1-fff4a4908ac9 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:55.070788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40f946-9f4a-4224-91f7-c00a0ee9e013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:55.071767 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40f946-9f4a-4224-91f7-c00a0ee9e013 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:55.072212 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40f946-9f4a-4224-91f7-c00a0ee9e013 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:55.072661 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40f946-9f4a-4224-91f7-c00a0ee9e013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:55.088612 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf40f946-9f4a-4224-91f7-c00a0ee9e013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:55.091332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf40f946-9f4a-4224-91f7-c00a0ee9e013 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:55.108995 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1345/2678] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:03:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:03:55.600280 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6453ce-6618-4831-89ac-dff3ef7487ed None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:55.601848 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1f6453ce-6618-4831-89ac-dff3ef7487ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:55.614828 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6453ce-6618-4831-89ac-dff3ef7487ed None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:55.622129 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8512f6f0-5764-49e8-b60f-e767fb65e4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:55.622769 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512f6f0-5764-49e8-b60f-e767fb65e4e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:55.623118 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512f6f0-5764-49e8-b60f-e767fb65e4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:55.623482 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512f6f0-5764-49e8-b60f-e767fb65e4e3 admin admin] PATH_INFO: `/v3/projects/cea308312ea240a0be12f37e10a93b33` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:55.624585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8512f6f0-5764-49e8-b60f-e767fb65e4e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cea308312ea240a0be12f37e10a93b33)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:55.626545 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8512f6f0-5764-49e8-b60f-e767fb65e4e3 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:55.645700 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:03:55.886009 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8512f6f0-5764-49e8-b60f-e767fb65e4e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cea308312ea240a0be12f37e10a93b33', 'request_id': 'req-8512f6f0-5764-49e8-b60f-e767fb65e4e3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:55.912899 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8512f6f0-5764-49e8-b60f-e767fb65e4e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cea308312ea240a0be12f37e10a93b33', 'request_id': 'req-8512f6f0-5764-49e8-b60f-e767fb65e4e3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:56.054142 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8512f6f0-5764-49e8-b60f-e767fb65e4e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cea308312ea240a0be12f37e10a93b33', 'request_id': 'req-8512f6f0-5764-49e8-b60f-e767fb65e4e3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:56.067592 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8512f6f0-5764-49e8-b60f-e767fb65e4e3 admin admin] The token cache is being invalidate because project cea308312ea240a0be12f37e10a93b33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:03:56.067888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8512f6f0-5764-49e8-b60f-e767fb65e4e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:56.068909 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8512f6f0-5764-49e8-b60f-e767fb65e4e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:03:56.070466 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1334/2679] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:03:55 2022] DELETE /identity/v3/projects/cea308312ea240a0be12f37e10a93b33 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:03:56.075816 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d016b7-33a2-4783-8180-b7c03f0d322f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:56.076267 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d016b7-33a2-4783-8180-b7c03f0d322f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:56.076589 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d016b7-33a2-4783-8180-b7c03f0d322f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:56.100844 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-21d016b7-33a2-4783-8180-b7c03f0d322f None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:03:56.102075 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-21d016b7-33a2-4783-8180-b7c03f0d322f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:56.173121 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1346/2680] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:03:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:03:56.183436 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b167cb8-e07e-4bfe-b14b-49a954f8b5b0 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:56.185392 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b167cb8-e07e-4bfe-b14b-49a954f8b5b0 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:56.185392 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b167cb8-e07e-4bfe-b14b-49a954f8b5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:56.204289 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9b167cb8-e07e-4bfe-b14b-49a954f8b5b0 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:03:56.205510 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9b167cb8-e07e-4bfe-b14b-49a954f8b5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:56.216131 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1335/2681] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:03:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:03:56.220752 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d016b7-33a2-4783-8180-b7c03f0d322f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:56.230558 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d016b7-33a2-4783-8180-b7c03f0d322f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:56.235208 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a032a3-5c33-4576-8bcc-77c59859e0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:56.235851 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a032a3-5c33-4576-8bcc-77c59859e0a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:56.236130 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a032a3-5c33-4576-8bcc-77c59859e0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:56.236404 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a032a3-5c33-4576-8bcc-77c59859e0a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:56.237096 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a032a3-5c33-4576-8bcc-77c59859e0a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:03:56.237403 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a032a3-5c33-4576-8bcc-77c59859e0a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:56.239618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a032a3-5c33-4576-8bcc-77c59859e0a8 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:56.251755 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1347/2682] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 15:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:03:56.257112 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b167cb8-e07e-4bfe-b14b-49a954f8b5b0 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:56.269150 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b167cb8-e07e-4bfe-b14b-49a954f8b5b0 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:56.277237 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77121196-1a27-49a1-acb8-4ba6f7d04c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:56.277901 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77121196-1a27-49a1-acb8-4ba6f7d04c83 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:56.278208 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77121196-1a27-49a1-acb8-4ba6f7d04c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:56.278580 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77121196-1a27-49a1-acb8-4ba6f7d04c83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:56.279261 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77121196-1a27-49a1-acb8-4ba6f7d04c83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:56.281325 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77121196-1a27-49a1-acb8-4ba6f7d04c83 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:56.465293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1336/2683] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 15:03:56 2022] POST /identity/v3/projects => generated 385 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:03:56.469974 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a032a3-5c33-4576-8bcc-77c59859e0a8 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:56.480181 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a032a3-5c33-4576-8bcc-77c59859e0a8 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:56.512972 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:03:56.541202 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e3deb-5273-4c1f-8370-f0c0546ba78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:56.541953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e3deb-5273-4c1f-8370-f0c0546ba78f admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:56.542326 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e3deb-5273-4c1f-8370-f0c0546ba78f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:56.542604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e3deb-5273-4c1f-8370-f0c0546ba78f admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:56.543314 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302e3deb-5273-4c1f-8370-f0c0546ba78f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:56.545389 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302e3deb-5273-4c1f-8370-f0c0546ba78f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:56.700193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1348/2684] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 15:03:56 2022] POST /identity/v3/users => generated 293 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:03:56.704528 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77121196-1a27-49a1-acb8-4ba6f7d04c83 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:56.715271 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77121196-1a27-49a1-acb8-4ba6f7d04c83 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:56.719947 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6976b81-a473-4c51-aef7-fcfc7899feaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:56.720573 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6976b81-a473-4c51-aef7-fcfc7899feaf admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:56.720855 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6976b81-a473-4c51-aef7-fcfc7899feaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:56.721178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6976b81-a473-4c51-aef7-fcfc7899feaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:56.721818 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6976b81-a473-4c51-aef7-fcfc7899feaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:56.723954 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6976b81-a473-4c51-aef7-fcfc7899feaf admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:56.740957 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1337/2685] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:03:56.744929 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e3deb-5273-4c1f-8370-f0c0546ba78f admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:56.756488 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e3deb-5273-4c1f-8370-f0c0546ba78f admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:56.762455 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e327d-3e49-4071-bbe4-4a6ad5e1c9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:56.763228 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e327d-3e49-4071-bbe4-4a6ad5e1c9c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:56.763725 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e327d-3e49-4071-bbe4-4a6ad5e1c9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:56.764004 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e327d-3e49-4071-bbe4-4a6ad5e1c9c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:56.765047 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e327d-3e49-4071-bbe4-4a6ad5e1c9c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:56.767632 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e327d-3e49-4071-bbe4-4a6ad5e1c9c5 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:56.783276 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1349/2686] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:03:56.788032 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6976b81-a473-4c51-aef7-fcfc7899feaf admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:56.799282 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6976b81-a473-4c51-aef7-fcfc7899feaf admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:56.804667 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc7841-559e-4f75-9746-a14901aa9233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:56.805358 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc7841-559e-4f75-9746-a14901aa9233 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:56.805760 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc7841-559e-4f75-9746-a14901aa9233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:56.812365 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc7841-559e-4f75-9746-a14901aa9233 admin admin] PATH_INFO: `/v3/projects/4498e62d24d244c98ec4d6f92950b4d5/users/a5eeb13648f844479d76dc50fe02811d/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:56.821217 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cc7841-559e-4f75-9746-a14901aa9233 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4498e62d24d244c98ec4d6f92950b4d5, user_id=a5eeb13648f844479d76dc50fe02811d, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:56.825399 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cc7841-559e-4f75-9746-a14901aa9233 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:56.954787 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1338/2687] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:03:56 2022] PUT /identity/v3/projects/4498e62d24d244c98ec4d6f92950b4d5/users/a5eeb13648f844479d76dc50fe02811d/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:03:57.028006 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f05635-1fdc-4aab-be7a-ac812ac0b850 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:57.028503 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f05635-1fdc-4aab-be7a-ac812ac0b850 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:57.028864 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f05635-1fdc-4aab-be7a-ac812ac0b850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:57.062639 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-52f05635-1fdc-4aab-be7a-ac812ac0b850 None None] MFA Rules not processed for user `a5eeb13648f844479d76dc50fe02811d`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:03:57.072645 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-52f05635-1fdc-4aab-be7a-ac812ac0b850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:57.135739 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1350/2688] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:03:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:03:57.146690 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc7841-559e-4f75-9746-a14901aa9233 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:57.148895 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b4cc7841-559e-4f75-9746-a14901aa9233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:57.211517 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc7841-559e-4f75-9746-a14901aa9233 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:57.222191 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d359bbb-74ad-4fb5-965e-10963a7f193b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:57.222191 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d359bbb-74ad-4fb5-965e-10963a7f193b service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:57.222191 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d359bbb-74ad-4fb5-965e-10963a7f193b service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:57.222191 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d359bbb-74ad-4fb5-965e-10963a7f193b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:57.226280 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d359bbb-74ad-4fb5-965e-10963a7f193b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:57.227773 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d359bbb-74ad-4fb5-965e-10963a7f193b service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:57.237560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1339/2689] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:03:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:04:03.354339 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f7da8-21a6-4a24-89bb-a60cf72bd076 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:03.355226 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f7da8-21a6-4a24-89bb-a60cf72bd076 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:03.355226 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f7da8-21a6-4a24-89bb-a60cf72bd076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:03.381000 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-672f7da8-21a6-4a24-89bb-a60cf72bd076 None None] MFA Rules not processed for user `5179c3c4eaaa41f3beb79ba7f1993478`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:03.382580 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-672f7da8-21a6-4a24-89bb-a60cf72bd076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:03.447425 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1351/2690] 10.222.0.97 () {58 vars in 1009 bytes} [Tue Jul 26 15:04:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:04:03.461121 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d359bbb-74ad-4fb5-965e-10963a7f193b service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:03.462536 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0d359bbb-74ad-4fb5-965e-10963a7f193b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:03.514648 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:03.538031 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d359bbb-74ad-4fb5-965e-10963a7f193b service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:03.543053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37bebdd-704e-44b4-bbab-1ae7ba2cff66 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85122c4052fd4ba19a2dae185a707caf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:03.543818 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bebdd-704e-44b4-bbab-1ae7ba2cff66 service swift] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:03.544104 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bebdd-704e-44b4-bbab-1ae7ba2cff66 service swift] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:03.544379 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bebdd-704e-44b4-bbab-1ae7ba2cff66 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:03.553206 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37bebdd-704e-44b4-bbab-1ae7ba2cff66 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:03.557528 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37bebdd-704e-44b4-bbab-1ae7ba2cff66 service swift] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:03.562927 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1340/2691] 10.222.0.97 () {60 vars in 1384 bytes} [Tue Jul 26 15:04:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:04:04.816787 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f7da8-21a6-4a24-89bb-a60cf72bd076 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:04.877290 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:04.893151 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f7da8-21a6-4a24-89bb-a60cf72bd076 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:04.899069 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5011e8e-09c8-4921-9962-ed06c4b71e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:04.899943 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5011e8e-09c8-4921-9962-ed06c4b71e76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:04.900336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5011e8e-09c8-4921-9962-ed06c4b71e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:04.900779 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5011e8e-09c8-4921-9962-ed06c4b71e76 admin admin] PATH_INFO: `/v3/users/a5eeb13648f844479d76dc50fe02811d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:04.901966 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5011e8e-09c8-4921-9962-ed06c4b71e76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5eeb13648f844479d76dc50fe02811d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:04.904106 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5011e8e-09c8-4921-9962-ed06c4b71e76 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:05.099714 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d5011e8e-09c8-4921-9962-ed06c4b71e76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5eeb13648f844479d76dc50fe02811d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:05.100690 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d5011e8e-09c8-4921-9962-ed06c4b71e76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5eeb13648f844479d76dc50fe02811d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:05.116633 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d5011e8e-09c8-4921-9962-ed06c4b71e76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5eeb13648f844479d76dc50fe02811d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:05.131871 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d5011e8e-09c8-4921-9962-ed06c4b71e76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5eeb13648f844479d76dc50fe02811d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:05.214441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d5011e8e-09c8-4921-9962-ed06c4b71e76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5eeb13648f844479d76dc50fe02811d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:05.494227 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1352/2692] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:04:04 2022] DELETE /identity/v3/users/a5eeb13648f844479d76dc50fe02811d => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:04:05.499720 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e02bd-2b83-46b8-a985-c70e43453a8e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:05.500955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e02bd-2b83-46b8-a985-c70e43453a8e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:05.501425 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e02bd-2b83-46b8-a985-c70e43453a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:05.533842 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-f41e02bd-2b83-46b8-a985-c70e43453a8e None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:05.536513 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f41e02bd-2b83-46b8-a985-c70e43453a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:05.620364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1341/2693] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:05.632909 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5011e8e-09c8-4921-9962-ed06c4b71e76 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:05.637655 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d5011e8e-09c8-4921-9962-ed06c4b71e76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:05.723119 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5011e8e-09c8-4921-9962-ed06c4b71e76 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:05.729059 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18112550-f8ca-4ed3-8d22-e61d030b528a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:05.729957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18112550-f8ca-4ed3-8d22-e61d030b528a service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:05.730279 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18112550-f8ca-4ed3-8d22-e61d030b528a service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:05.730559 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18112550-f8ca-4ed3-8d22-e61d030b528a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:05.743719 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18112550-f8ca-4ed3-8d22-e61d030b528a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:05.745237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18112550-f8ca-4ed3-8d22-e61d030b528a service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:05.762956 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1353/2694] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:04:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:04:06.273020 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41e02bd-2b83-46b8-a985-c70e43453a8e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:06.273020 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f41e02bd-2b83-46b8-a985-c70e43453a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:06.287710 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41e02bd-2b83-46b8-a985-c70e43453a8e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:06.294599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc18839-4857-4ca0-b70c-ea87cad3d1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:06.295573 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc18839-4857-4ca0-b70c-ea87cad3d1ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:06.295861 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc18839-4857-4ca0-b70c-ea87cad3d1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:06.296989 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc18839-4857-4ca0-b70c-ea87cad3d1ea admin admin] PATH_INFO: `/v3/projects/4498e62d24d244c98ec4d6f92950b4d5` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:06.297724 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc18839-4857-4ca0-b70c-ea87cad3d1ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=4498e62d24d244c98ec4d6f92950b4d5)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:06.300071 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc18839-4857-4ca0-b70c-ea87cad3d1ea admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:06.742663 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2bc18839-4857-4ca0-b70c-ea87cad3d1ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4498e62d24d244c98ec4d6f92950b4d5', 'request_id': 'req-2bc18839-4857-4ca0-b70c-ea87cad3d1ea'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:06.748513 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2bc18839-4857-4ca0-b70c-ea87cad3d1ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4498e62d24d244c98ec4d6f92950b4d5', 'request_id': 'req-2bc18839-4857-4ca0-b70c-ea87cad3d1ea'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:06.837308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2bc18839-4857-4ca0-b70c-ea87cad3d1ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4498e62d24d244c98ec4d6f92950b4d5', 'request_id': 'req-2bc18839-4857-4ca0-b70c-ea87cad3d1ea'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:06.849122 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2bc18839-4857-4ca0-b70c-ea87cad3d1ea admin admin] The token cache is being invalidate because project 4498e62d24d244c98ec4d6f92950b4d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:04:06.849417 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2bc18839-4857-4ca0-b70c-ea87cad3d1ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:06.850093 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2bc18839-4857-4ca0-b70c-ea87cad3d1ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:06.851904 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1342/2695] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:04:06 2022] DELETE /identity/v3/projects/4498e62d24d244c98ec4d6f92950b4d5 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:04:06.856947 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4af7f-e45a-4199-b70b-071c81d719b9 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:06.857802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4af7f-e45a-4199-b70b-071c81d719b9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:06.858148 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4af7f-e45a-4199-b70b-071c81d719b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:06.882082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d3e4af7f-e45a-4199-b70b-071c81d719b9 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:06.885684 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d3e4af7f-e45a-4199-b70b-071c81d719b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:06.971449 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1354/2696] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:04:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:06.976762 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba25fca-a4e0-4873-88c7-abf37c2d3cb4 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:06.977055 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba25fca-a4e0-4873-88c7-abf37c2d3cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:06.977338 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba25fca-a4e0-4873-88c7-abf37c2d3cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:06.998135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6ba25fca-a4e0-4873-88c7-abf37c2d3cb4 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:06.999305 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6ba25fca-a4e0-4873-88c7-abf37c2d3cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:07.007390 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1343/2697] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:04:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:07.013429 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4af7f-e45a-4199-b70b-071c81d719b9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:07.024932 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4af7f-e45a-4199-b70b-071c81d719b9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:07.030315 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc8dc8e-4346-4ed8-ba8d-8bc45141b1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:07.030926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8dc8e-4346-4ed8-ba8d-8bc45141b1dc admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:07.031264 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8dc8e-4346-4ed8-ba8d-8bc45141b1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:07.031608 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8dc8e-4346-4ed8-ba8d-8bc45141b1dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:07.032286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc8dc8e-4346-4ed8-ba8d-8bc45141b1dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:04:07.032615 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc8dc8e-4346-4ed8-ba8d-8bc45141b1dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:07.035577 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc8dc8e-4346-4ed8-ba8d-8bc45141b1dc admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:07.045075 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:07.050280 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1355/2698] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 15:04:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:04:07.054249 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba25fca-a4e0-4873-88c7-abf37c2d3cb4 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:07.066259 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba25fca-a4e0-4873-88c7-abf37c2d3cb4 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:07.071309 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96f224-0a12-403a-aabc-3b790bb522fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:07.071981 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96f224-0a12-403a-aabc-3b790bb522fd admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:07.072267 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96f224-0a12-403a-aabc-3b790bb522fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:07.072544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96f224-0a12-403a-aabc-3b790bb522fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:07.073210 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f96f224-0a12-403a-aabc-3b790bb522fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:07.075118 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f96f224-0a12-403a-aabc-3b790bb522fd admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:07.204768 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1344/2699] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 15:04:07 2022] POST /identity/v3/projects => generated 373 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:04:07.211194 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc8dc8e-4346-4ed8-ba8d-8bc45141b1dc admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:07.222601 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc8dc8e-4346-4ed8-ba8d-8bc45141b1dc admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:07.302650 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805698e-43d9-447e-b751-89743f0532cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:07.303881 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805698e-43d9-447e-b751-89743f0532cb admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:07.304494 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805698e-43d9-447e-b751-89743f0532cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:07.304945 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805698e-43d9-447e-b751-89743f0532cb admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:07.305786 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d805698e-43d9-447e-b751-89743f0532cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:07.307928 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d805698e-43d9-447e-b751-89743f0532cb admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:07.411956 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1356/2700] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 15:04:07 2022] POST /identity/v3/users => generated 287 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:04:07.413880 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96f224-0a12-403a-aabc-3b790bb522fd admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:07.425948 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96f224-0a12-403a-aabc-3b790bb522fd admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:07.433461 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c143c9-00f0-4a56-8103-ed28b4f8cf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:07.434360 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c143c9-00f0-4a56-8103-ed28b4f8cf8f admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:07.434827 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c143c9-00f0-4a56-8103-ed28b4f8cf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:07.435093 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c143c9-00f0-4a56-8103-ed28b4f8cf8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:07.435885 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c143c9-00f0-4a56-8103-ed28b4f8cf8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:07.437508 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c143c9-00f0-4a56-8103-ed28b4f8cf8f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:07.452337 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1345/2701] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:07.458151 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805698e-43d9-447e-b751-89743f0532cb admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:07.474796 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805698e-43d9-447e-b751-89743f0532cb admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:07.481310 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494afa15-2b88-4fc0-930d-a01ed30f7b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:07.482134 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494afa15-2b88-4fc0-930d-a01ed30f7b67 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:07.482522 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494afa15-2b88-4fc0-930d-a01ed30f7b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:07.482902 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494afa15-2b88-4fc0-930d-a01ed30f7b67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:07.483650 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494afa15-2b88-4fc0-930d-a01ed30f7b67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:07.485268 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494afa15-2b88-4fc0-930d-a01ed30f7b67 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:07.505039 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c143c9-00f0-4a56-8103-ed28b4f8cf8f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:07.505263 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1357/2702] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:07.521410 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c143c9-00f0-4a56-8103-ed28b4f8cf8f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:07.527647 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10d9da-92cf-49f5-bc5e-53e58d7c3eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:07.528646 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10d9da-92cf-49f5-bc5e-53e58d7c3eff admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:07.529006 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10d9da-92cf-49f5-bc5e-53e58d7c3eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:07.529419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10d9da-92cf-49f5-bc5e-53e58d7c3eff admin admin] PATH_INFO: `/v3/projects/ad1bd5580c6b4dbbb84cd0969130828d/users/5111d8fb11e548f9a7bf0c8d3639b24a/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:07.551259 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa10d9da-92cf-49f5-bc5e-53e58d7c3eff admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad1bd5580c6b4dbbb84cd0969130828d, user_id=5111d8fb11e548f9a7bf0c8d3639b24a, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:07.556665 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa10d9da-92cf-49f5-bc5e-53e58d7c3eff admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:07.560207 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:07.644754 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1346/2703] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:04:07 2022] PUT /identity/v3/projects/ad1bd5580c6b4dbbb84cd0969130828d/users/5111d8fb11e548f9a7bf0c8d3639b24a/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:04:07.667800 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18895d1-2836-42cc-a980-149139719445 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:07.668570 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18895d1-2836-42cc-a980-149139719445 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:07.669321 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18895d1-2836-42cc-a980-149139719445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:07.708188 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-b18895d1-2836-42cc-a980-149139719445 None None] MFA Rules not processed for user `5111d8fb11e548f9a7bf0c8d3639b24a`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:07.709572 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b18895d1-2836-42cc-a980-149139719445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:07.798077 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1358/2704] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:04:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:07.810560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10d9da-92cf-49f5-bc5e-53e58d7c3eff admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:07.813180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-fa10d9da-92cf-49f5-bc5e-53e58d7c3eff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:07.878550 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10d9da-92cf-49f5-bc5e-53e58d7c3eff admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:07.882653 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26cc0ac-b5c1-4fd8-9cb2-ab680c93f2bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:07.883277 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cc0ac-b5c1-4fd8-9cb2-ab680c93f2bb service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:07.883591 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cc0ac-b5c1-4fd8-9cb2-ab680c93f2bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:07.883871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cc0ac-b5c1-4fd8-9cb2-ab680c93f2bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:07.897247 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26cc0ac-b5c1-4fd8-9cb2-ab680c93f2bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:07.898866 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26cc0ac-b5c1-4fd8-9cb2-ab680c93f2bb service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:07.912297 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1347/2705] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:04:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:04:17.794286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=67,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:17.904741 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:20.337321 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb86193-7300-4110-ae5f-605fcfa32328 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:20.337862 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb86193-7300-4110-ae5f-605fcfa32328 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:20.337967 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb86193-7300-4110-ae5f-605fcfa32328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:20.358861 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-adb86193-7300-4110-ae5f-605fcfa32328 None None] MFA Rules not processed for user `5179c3c4eaaa41f3beb79ba7f1993478`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:20.360173 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-adb86193-7300-4110-ae5f-605fcfa32328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:20.418168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1359/2706] 10.222.0.97 () {58 vars in 967 bytes} [Tue Jul 26 15:04:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:20.430940 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26cc0ac-b5c1-4fd8-9cb2-ab680c93f2bb service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:20.432376 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b26cc0ac-b5c1-4fd8-9cb2-ab680c93f2bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:20.503442 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26cc0ac-b5c1-4fd8-9cb2-ab680c93f2bb service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:20.508311 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f08a4-10dc-490a-b3db-ac603b143082 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85122c4052fd4ba19a2dae185a707caf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:20.508935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f08a4-10dc-490a-b3db-ac603b143082 service swift] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:20.509219 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f08a4-10dc-490a-b3db-ac603b143082 service swift] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:20.509501 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f08a4-10dc-490a-b3db-ac603b143082 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:20.518247 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40f08a4-10dc-490a-b3db-ac603b143082 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:20.519869 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40f08a4-10dc-490a-b3db-ac603b143082 service swift] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:20.528107 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1348/2707] 10.222.0.97 () {60 vars in 1384 bytes} [Tue Jul 26 15:04:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:04:23.850043 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb86193-7300-4110-ae5f-605fcfa32328 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:23.851651 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-adb86193-7300-4110-ae5f-605fcfa32328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:23.919560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f08a4-10dc-490a-b3db-ac603b143082 service swift] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:23.931045 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb86193-7300-4110-ae5f-605fcfa32328 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:23.937359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b652500b-87db-4c68-9aba-b9c3ae108ab4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '252c2d2939134416a3e8efb254f03c5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:23.938124 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652500b-87db-4c68-9aba-b9c3ae108ab4 service placement] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:23.938415 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652500b-87db-4c68-9aba-b9c3ae108ab4 service placement] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:23.938746 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652500b-87db-4c68-9aba-b9c3ae108ab4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:23.941664 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b652500b-87db-4c68-9aba-b9c3ae108ab4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:23.953854 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b652500b-87db-4c68-9aba-b9c3ae108ab4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:23.955992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b652500b-87db-4c68-9aba-b9c3ae108ab4 service placement] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:23.966762 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1360/2708] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:04:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:04:23.988389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f08a4-10dc-490a-b3db-ac603b143082 service swift] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:23.993803 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7ee2a-5296-4a01-97e7-114959aa7eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:23.994568 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7ee2a-5296-4a01-97e7-114959aa7eaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:23.994851 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7ee2a-5296-4a01-97e7-114959aa7eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:23.995125 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7ee2a-5296-4a01-97e7-114959aa7eaf admin admin] PATH_INFO: `/v3/users/5111d8fb11e548f9a7bf0c8d3639b24a` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:23.996421 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd7ee2a-5296-4a01-97e7-114959aa7eaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=5111d8fb11e548f9a7bf0c8d3639b24a)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:23.998429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd7ee2a-5296-4a01-97e7-114959aa7eaf admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:24.012202 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b652500b-87db-4c68-9aba-b9c3ae108ab4 service placement] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:24.014533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b652500b-87db-4c68-9aba-b9c3ae108ab4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:24.030833 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b652500b-87db-4c68-9aba-b9c3ae108ab4 service placement] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:24.036241 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34decaeb-e668-4d4c-b7c2-813826cc186b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:24.036897 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34decaeb-e668-4d4c-b7c2-813826cc186b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:24.037186 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34decaeb-e668-4d4c-b7c2-813826cc186b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:24.037473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34decaeb-e668-4d4c-b7c2-813826cc186b admin admin] PATH_INFO: `/v3/users/39d959e5dbd346708e87b496d344af28` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:24.038665 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34decaeb-e668-4d4c-b7c2-813826cc186b admin admin] RBAC: Authorizing `identity:delete_user(user_id=39d959e5dbd346708e87b496d344af28)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:24.040639 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34decaeb-e668-4d4c-b7c2-813826cc186b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:24.316962 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9cd7ee2a-5296-4a01-97e7-114959aa7eaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5111d8fb11e548f9a7bf0c8d3639b24a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:24.317620 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9cd7ee2a-5296-4a01-97e7-114959aa7eaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5111d8fb11e548f9a7bf0c8d3639b24a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:24.329383 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9cd7ee2a-5296-4a01-97e7-114959aa7eaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5111d8fb11e548f9a7bf0c8d3639b24a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:24.337094 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9cd7ee2a-5296-4a01-97e7-114959aa7eaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5111d8fb11e548f9a7bf0c8d3639b24a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:24.337733 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9cd7ee2a-5296-4a01-97e7-114959aa7eaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5111d8fb11e548f9a7bf0c8d3639b24a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:24.362415 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-34decaeb-e668-4d4c-b7c2-813826cc186b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39d959e5dbd346708e87b496d344af28'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:24.363079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-34decaeb-e668-4d4c-b7c2-813826cc186b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39d959e5dbd346708e87b496d344af28'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:24.375175 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-34decaeb-e668-4d4c-b7c2-813826cc186b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39d959e5dbd346708e87b496d344af28'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:24.382842 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-34decaeb-e668-4d4c-b7c2-813826cc186b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39d959e5dbd346708e87b496d344af28'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:24.383549 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-34decaeb-e668-4d4c-b7c2-813826cc186b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39d959e5dbd346708e87b496d344af28'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:24.441274 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1349/2709] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:04:23 2022] DELETE /identity/v3/users/5111d8fb11e548f9a7bf0c8d3639b24a => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:04:24.445507 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7da505-a978-420f-9b21-05721757eff2 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:24.445813 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7da505-a978-420f-9b21-05721757eff2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:24.446080 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7da505-a978-420f-9b21-05721757eff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:24.465178 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3a7da505-a978-420f-9b21-05721757eff2 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:24.466383 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3a7da505-a978-420f-9b21-05721757eff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:24.470322 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:24.525369 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1350/2710] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:04:24.535708 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7da505-a978-420f-9b21-05721757eff2 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:24.537059 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3a7da505-a978-420f-9b21-05721757eff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:24.554211 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1361/2711] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:04:24 2022] DELETE /identity/v3/users/39d959e5dbd346708e87b496d344af28 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:04:24.596611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7da505-a978-420f-9b21-05721757eff2 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:24.645990 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1110c3eb-cc02-43eb-b805-42b3c5ba5f5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:24.646898 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110c3eb-cc02-43eb-b805-42b3c5ba5f5c service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:24.647187 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110c3eb-cc02-43eb-b805-42b3c5ba5f5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:24.647620 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110c3eb-cc02-43eb-b805-42b3c5ba5f5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:24.711673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1110c3eb-cc02-43eb-b805-42b3c5ba5f5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:24.713182 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1110c3eb-cc02-43eb-b805-42b3c5ba5f5c service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:24.722957 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1351/2712] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:04:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:04:24.893550 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34decaeb-e668-4d4c-b7c2-813826cc186b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:24.895185 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-34decaeb-e668-4d4c-b7c2-813826cc186b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:24.908882 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34decaeb-e668-4d4c-b7c2-813826cc186b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:24.915299 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193fc433-1178-40e7-b533-4d51c7835af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:24.916769 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193fc433-1178-40e7-b533-4d51c7835af1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:24.917064 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193fc433-1178-40e7-b533-4d51c7835af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:24.917519 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193fc433-1178-40e7-b533-4d51c7835af1 admin admin] PATH_INFO: `/v3/projects/ba1773de981b41bd829bc90d92f2f649` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:24.918740 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193fc433-1178-40e7-b533-4d51c7835af1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba1773de981b41bd829bc90d92f2f649)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:24.920720 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193fc433-1178-40e7-b533-4d51c7835af1 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:24.922935 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:25.085594 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-193fc433-1178-40e7-b533-4d51c7835af1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba1773de981b41bd829bc90d92f2f649', 'request_id': 'req-193fc433-1178-40e7-b533-4d51c7835af1'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:25.092278 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-193fc433-1178-40e7-b533-4d51c7835af1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba1773de981b41bd829bc90d92f2f649', 'request_id': 'req-193fc433-1178-40e7-b533-4d51c7835af1'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:25.229303 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-193fc433-1178-40e7-b533-4d51c7835af1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba1773de981b41bd829bc90d92f2f649', 'request_id': 'req-193fc433-1178-40e7-b533-4d51c7835af1'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:25.241617 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-193fc433-1178-40e7-b533-4d51c7835af1 admin admin] The token cache is being invalidate because project ba1773de981b41bd829bc90d92f2f649 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:04:25.241934 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-193fc433-1178-40e7-b533-4d51c7835af1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:25.242690 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-193fc433-1178-40e7-b533-4d51c7835af1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:25.244233 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1362/2713] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:04:24 2022] DELETE /identity/v3/projects/ba1773de981b41bd829bc90d92f2f649 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:04:25.250033 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6e738-9970-48c3-853d-0742d84bd887 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:25.250540 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6e738-9970-48c3-853d-0742d84bd887 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:25.250635 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6e738-9970-48c3-853d-0742d84bd887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:25.270415 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-79b6e738-9970-48c3-853d-0742d84bd887 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:25.271943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-79b6e738-9970-48c3-853d-0742d84bd887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:25.281376 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:25.330700 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1352/2714] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:04:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:25.338425 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f427a6-6851-4e89-8c9c-5d275bcef015 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:25.338798 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f427a6-6851-4e89-8c9c-5d275bcef015 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:25.339087 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f427a6-6851-4e89-8c9c-5d275bcef015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:25.349426 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6e738-9970-48c3-853d-0742d84bd887 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:25.369904 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-79b6e738-9970-48c3-853d-0742d84bd887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:25.371376 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-e9f427a6-6851-4e89-8c9c-5d275bcef015 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:25.371376 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e9f427a6-6851-4e89-8c9c-5d275bcef015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:25.371712 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6e738-9970-48c3-853d-0742d84bd887 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:25.375847 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9a1c5-9811-4e3c-bc9b-800a3554bd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:25.376548 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9a1c5-9811-4e3c-bc9b-800a3554bd5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:25.377189 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9a1c5-9811-4e3c-bc9b-800a3554bd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:25.377189 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9a1c5-9811-4e3c-bc9b-800a3554bd5c admin admin] PATH_INFO: `/v3/projects/ad1bd5580c6b4dbbb84cd0969130828d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:25.378543 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b9a1c5-9811-4e3c-bc9b-800a3554bd5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad1bd5580c6b4dbbb84cd0969130828d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:25.380530 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b9a1c5-9811-4e3c-bc9b-800a3554bd5c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:25.388787 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1363/2715] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:04:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:25.393053 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f427a6-6851-4e89-8c9c-5d275bcef015 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:25.412077 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f427a6-6851-4e89-8c9c-5d275bcef015 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:25.417125 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820a655-8170-4ad5-a150-507bc6f71b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:25.417722 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820a655-8170-4ad5-a150-507bc6f71b80 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:25.418012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820a655-8170-4ad5-a150-507bc6f71b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:25.418293 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820a655-8170-4ad5-a150-507bc6f71b80 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:25.419005 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b820a655-8170-4ad5-a150-507bc6f71b80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:04:25.419306 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b820a655-8170-4ad5-a150-507bc6f71b80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:25.420939 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b820a655-8170-4ad5-a150-507bc6f71b80 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:25.433619 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1364/2716] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 15:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:04:25.441519 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820a655-8170-4ad5-a150-507bc6f71b80 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:25.451606 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820a655-8170-4ad5-a150-507bc6f71b80 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:25.456684 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55101eb-ff65-42b2-889c-cf4267728abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:25.457283 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55101eb-ff65-42b2-889c-cf4267728abf admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:25.457562 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55101eb-ff65-42b2-889c-cf4267728abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:25.457837 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55101eb-ff65-42b2-889c-cf4267728abf admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:25.458489 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55101eb-ff65-42b2-889c-cf4267728abf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:25.460421 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55101eb-ff65-42b2-889c-cf4267728abf admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:25.532943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e0b9a1c5-9811-4e3c-bc9b-800a3554bd5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad1bd5580c6b4dbbb84cd0969130828d', 'request_id': 'req-e0b9a1c5-9811-4e3c-bc9b-800a3554bd5c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:25.539871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e0b9a1c5-9811-4e3c-bc9b-800a3554bd5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad1bd5580c6b4dbbb84cd0969130828d', 'request_id': 'req-e0b9a1c5-9811-4e3c-bc9b-800a3554bd5c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:25.643529 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1365/2717] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 15:04:25 2022] POST /identity/v3/projects => generated 373 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:04:25.647587 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55101eb-ff65-42b2-889c-cf4267728abf admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:25.689784 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e0b9a1c5-9811-4e3c-bc9b-800a3554bd5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad1bd5580c6b4dbbb84cd0969130828d', 'request_id': 'req-e0b9a1c5-9811-4e3c-bc9b-800a3554bd5c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:25.703087 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e0b9a1c5-9811-4e3c-bc9b-800a3554bd5c admin admin] The token cache is being invalidate because project ad1bd5580c6b4dbbb84cd0969130828d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:04:25.703379 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e0b9a1c5-9811-4e3c-bc9b-800a3554bd5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:25.719359 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e0b9a1c5-9811-4e3c-bc9b-800a3554bd5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:25.719359 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1353/2718] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:04:25 2022] DELETE /identity/v3/projects/ad1bd5580c6b4dbbb84cd0969130828d => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:04:25.719652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55101eb-ff65-42b2-889c-cf4267728abf admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:25.719652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fdea90-a940-400e-9150-3c0f76817b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:25.719652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdea90-a940-400e-9150-3c0f76817b95 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:25.719652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdea90-a940-400e-9150-3c0f76817b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:25.719990 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdea90-a940-400e-9150-3c0f76817b95 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:25.719990 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fdea90-a940-400e-9150-3c0f76817b95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:25.719990 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fdea90-a940-400e-9150-3c0f76817b95 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:25.830456 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1366/2719] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 15:04:25 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:04:25.834902 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9a1c5-9811-4e3c-bc9b-800a3554bd5c admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:25.836638 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e0b9a1c5-9811-4e3c-bc9b-800a3554bd5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:25.911685 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9a1c5-9811-4e3c-bc9b-800a3554bd5c admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:25.916611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364861b-4937-4696-ac11-dc26f7f770a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:25.917285 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364861b-4937-4696-ac11-dc26f7f770a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:25.917566 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364861b-4937-4696-ac11-dc26f7f770a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:25.917844 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364861b-4937-4696-ac11-dc26f7f770a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:25.918622 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8364861b-4937-4696-ac11-dc26f7f770a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:25.920411 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8364861b-4937-4696-ac11-dc26f7f770a1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:25.932213 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1354/2720] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:25.936433 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fdea90-a940-400e-9150-3c0f76817b95 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:25.947465 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fdea90-a940-400e-9150-3c0f76817b95 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:25.952675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba6968-7f59-4d97-88d1-488419464e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:25.953322 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba6968-7f59-4d97-88d1-488419464e92 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:25.953656 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba6968-7f59-4d97-88d1-488419464e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:25.953936 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba6968-7f59-4d97-88d1-488419464e92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:25.954642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba6968-7f59-4d97-88d1-488419464e92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:25.956346 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba6968-7f59-4d97-88d1-488419464e92 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:25.967295 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1367/2721] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:25.972174 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364861b-4937-4696-ac11-dc26f7f770a1 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:25.982549 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364861b-4937-4696-ac11-dc26f7f770a1 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:25.987631 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28c755-6e17-4462-a34c-baee396148d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:25.988299 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28c755-6e17-4462-a34c-baee396148d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:25.988581 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28c755-6e17-4462-a34c-baee396148d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:25.988860 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28c755-6e17-4462-a34c-baee396148d7 admin admin] PATH_INFO: `/v3/projects/82d5ac21e5a241869e72087e6655aeb0/users/09a9ff1994d3406e9d6452bf73d27b69/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:26.003897 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e28c755-6e17-4462-a34c-baee396148d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82d5ac21e5a241869e72087e6655aeb0, user_id=09a9ff1994d3406e9d6452bf73d27b69, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:26.006601 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e28c755-6e17-4462-a34c-baee396148d7 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:26.126795 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1355/2722] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:04:25 2022] PUT /identity/v3/projects/82d5ac21e5a241869e72087e6655aeb0/users/09a9ff1994d3406e9d6452bf73d27b69/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:04:26.137683 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c52a3f-d108-4f99-86eb-ba55cc477ccf None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:26.137996 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c52a3f-d108-4f99-86eb-ba55cc477ccf None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:26.138277 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c52a3f-d108-4f99-86eb-ba55cc477ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:26.166710 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-e3c52a3f-d108-4f99-86eb-ba55cc477ccf None None] MFA Rules not processed for user `09a9ff1994d3406e9d6452bf73d27b69`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:26.167962 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e3c52a3f-d108-4f99-86eb-ba55cc477ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:26.181210 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:26.233335 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1368/2723] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:04:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:26.244464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28c755-6e17-4462-a34c-baee396148d7 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:26.246285 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6e28c755-6e17-4462-a34c-baee396148d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:26.259906 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:26.304898 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28c755-6e17-4462-a34c-baee396148d7 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:26.308918 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57efee4c-c3a0-42c7-bc28-b192e1e749ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:26.312379 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efee4c-c3a0-42c7-bc28-b192e1e749ca service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:26.312379 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efee4c-c3a0-42c7-bc28-b192e1e749ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:26.312379 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efee4c-c3a0-42c7-bc28-b192e1e749ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:26.318835 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57efee4c-c3a0-42c7-bc28-b192e1e749ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:26.320308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57efee4c-c3a0-42c7-bc28-b192e1e749ca service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:26.329077 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1356/2724] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:04:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:04:26.354765 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c52a3f-d108-4f99-86eb-ba55cc477ccf None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:26.416379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c52a3f-d108-4f99-86eb-ba55cc477ccf None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:26.421000 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730d868-f170-49a7-a4df-9d249d21f1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:26.421660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730d868-f170-49a7-a4df-9d249d21f1c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:26.421944 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730d868-f170-49a7-a4df-9d249d21f1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:26.422224 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730d868-f170-49a7-a4df-9d249d21f1c1 admin admin] PATH_INFO: `/v3/users/09a9ff1994d3406e9d6452bf73d27b69` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:26.423540 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730d868-f170-49a7-a4df-9d249d21f1c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09a9ff1994d3406e9d6452bf73d27b69)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:26.425968 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730d868-f170-49a7-a4df-9d249d21f1c1 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:26.671536 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b730d868-f170-49a7-a4df-9d249d21f1c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09a9ff1994d3406e9d6452bf73d27b69'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:26.672194 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b730d868-f170-49a7-a4df-9d249d21f1c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09a9ff1994d3406e9d6452bf73d27b69'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:26.684587 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b730d868-f170-49a7-a4df-9d249d21f1c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09a9ff1994d3406e9d6452bf73d27b69'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:26.691893 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b730d868-f170-49a7-a4df-9d249d21f1c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09a9ff1994d3406e9d6452bf73d27b69'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:26.692558 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b730d868-f170-49a7-a4df-9d249d21f1c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09a9ff1994d3406e9d6452bf73d27b69'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:27.050803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1369/2725] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:04:26 2022] DELETE /identity/v3/users/09a9ff1994d3406e9d6452bf73d27b69 => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:04:27.055158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b4aa5-10af-4f56-afa1-0dce467b309e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:27.055481 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b4aa5-10af-4f56-afa1-0dce467b309e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:27.055764 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b4aa5-10af-4f56-afa1-0dce467b309e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:27.074668 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ef3b4aa5-10af-4f56-afa1-0dce467b309e None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:27.075875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ef3b4aa5-10af-4f56-afa1-0dce467b309e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:27.135838 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1357/2726] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:04:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:27.147061 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730d868-f170-49a7-a4df-9d249d21f1c1 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:27.156363 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b730d868-f170-49a7-a4df-9d249d21f1c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:27.190248 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:27.209097 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730d868-f170-49a7-a4df-9d249d21f1c1 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:27.213056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb907f8a-af29-4890-a3c1-66f07e7d9fa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:27.213679 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb907f8a-af29-4890-a3c1-66f07e7d9fa7 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:27.213988 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb907f8a-af29-4890-a3c1-66f07e7d9fa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:27.214269 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb907f8a-af29-4890-a3c1-66f07e7d9fa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:27.223491 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb907f8a-af29-4890-a3c1-66f07e7d9fa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:27.224978 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb907f8a-af29-4890-a3c1-66f07e7d9fa7 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:27.234250 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1370/2727] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:04:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:04:27.699539 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b4aa5-10af-4f56-afa1-0dce467b309e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:27.705350 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ef3b4aa5-10af-4f56-afa1-0dce467b309e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:27.714612 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b4aa5-10af-4f56-afa1-0dce467b309e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:27.719587 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745eca7d-fc71-4178-9fe8-f4fe2144b0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:27.720272 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745eca7d-fc71-4178-9fe8-f4fe2144b0a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:27.721673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745eca7d-fc71-4178-9fe8-f4fe2144b0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:27.721966 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745eca7d-fc71-4178-9fe8-f4fe2144b0a2 admin admin] PATH_INFO: `/v3/projects/82d5ac21e5a241869e72087e6655aeb0` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:27.723387 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745eca7d-fc71-4178-9fe8-f4fe2144b0a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82d5ac21e5a241869e72087e6655aeb0)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:27.725419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745eca7d-fc71-4178-9fe8-f4fe2144b0a2 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:27.842100 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-745eca7d-fc71-4178-9fe8-f4fe2144b0a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82d5ac21e5a241869e72087e6655aeb0', 'request_id': 'req-745eca7d-fc71-4178-9fe8-f4fe2144b0a2'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:27.848366 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-745eca7d-fc71-4178-9fe8-f4fe2144b0a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82d5ac21e5a241869e72087e6655aeb0', 'request_id': 'req-745eca7d-fc71-4178-9fe8-f4fe2144b0a2'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:27.955051 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-745eca7d-fc71-4178-9fe8-f4fe2144b0a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82d5ac21e5a241869e72087e6655aeb0', 'request_id': 'req-745eca7d-fc71-4178-9fe8-f4fe2144b0a2'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:27.966504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-745eca7d-fc71-4178-9fe8-f4fe2144b0a2 admin admin] The token cache is being invalidate because project 82d5ac21e5a241869e72087e6655aeb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:04:27.966791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-745eca7d-fc71-4178-9fe8-f4fe2144b0a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:27.967441 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-745eca7d-fc71-4178-9fe8-f4fe2144b0a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:27.968942 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1358/2728] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:04:27 2022] DELETE /identity/v3/projects/82d5ac21e5a241869e72087e6655aeb0 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:04:27.973917 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3a672-5be8-40b0-96ac-312d3b25cc33 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:27.974209 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3a672-5be8-40b0-96ac-312d3b25cc33 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:27.974490 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3a672-5be8-40b0-96ac-312d3b25cc33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:27.994353 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-24b3a672-5be8-40b0-96ac-312d3b25cc33 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:27.995593 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-24b3a672-5be8-40b0-96ac-312d3b25cc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:28.054704 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1371/2729] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:04:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:28.062465 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf7cfa-ca8b-4023-8d89-08c945233b90 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:28.062759 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf7cfa-ca8b-4023-8d89-08c945233b90 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:28.063042 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf7cfa-ca8b-4023-8d89-08c945233b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:28.083206 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-1ddf7cfa-ca8b-4023-8d89-08c945233b90 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:28.084441 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1ddf7cfa-ca8b-4023-8d89-08c945233b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:28.093496 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1359/2730] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:04:28.105127 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b3a672-5be8-40b0-96ac-312d3b25cc33 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:28.115430 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b3a672-5be8-40b0-96ac-312d3b25cc33 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:28.119985 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc160d3-221f-4a53-93e2-dd50db14c9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:28.120596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc160d3-221f-4a53-93e2-dd50db14c9d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:28.120875 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc160d3-221f-4a53-93e2-dd50db14c9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:28.121170 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc160d3-221f-4a53-93e2-dd50db14c9d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:28.121853 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc160d3-221f-4a53-93e2-dd50db14c9d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:04:28.122157 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc160d3-221f-4a53-93e2-dd50db14c9d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:28.123774 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc160d3-221f-4a53-93e2-dd50db14c9d4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:28.135668 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1372/2731] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 15:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:04:28.142808 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf7cfa-ca8b-4023-8d89-08c945233b90 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:28.152665 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf7cfa-ca8b-4023-8d89-08c945233b90 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:28.157460 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf1716-854c-44c7-a841-54abd18bad2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:28.158574 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf1716-854c-44c7-a841-54abd18bad2b admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:28.158859 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf1716-854c-44c7-a841-54abd18bad2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:28.159138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf1716-854c-44c7-a841-54abd18bad2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:28.159834 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf1716-854c-44c7-a841-54abd18bad2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:28.161735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf1716-854c-44c7-a841-54abd18bad2b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:28.259131 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1360/2732] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 15:04:28 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:04:28.263266 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc160d3-221f-4a53-93e2-dd50db14c9d4 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:28.278139 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc160d3-221f-4a53-93e2-dd50db14c9d4 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:28.335662 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342cbb94-3649-4816-b9d3-c7c1fee327cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:28.336321 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342cbb94-3649-4816-b9d3-c7c1fee327cf admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:28.336609 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342cbb94-3649-4816-b9d3-c7c1fee327cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:28.336889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342cbb94-3649-4816-b9d3-c7c1fee327cf admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:28.337560 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342cbb94-3649-4816-b9d3-c7c1fee327cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:28.339516 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342cbb94-3649-4816-b9d3-c7c1fee327cf admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:28.423626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1373/2733] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 15:04:28 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:04:28.427818 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf1716-854c-44c7-a841-54abd18bad2b admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:28.438232 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf1716-854c-44c7-a841-54abd18bad2b admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:28.443391 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc6be8a-0dcd-4428-85ab-a05b5af74b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:28.444056 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6be8a-0dcd-4428-85ab-a05b5af74b01 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:28.444324 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6be8a-0dcd-4428-85ab-a05b5af74b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:28.444602 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6be8a-0dcd-4428-85ab-a05b5af74b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:28.445290 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc6be8a-0dcd-4428-85ab-a05b5af74b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:28.446881 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc6be8a-0dcd-4428-85ab-a05b5af74b01 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:28.449117 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:28.458057 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1361/2734] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:28.462417 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342cbb94-3649-4816-b9d3-c7c1fee327cf admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:28.472377 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342cbb94-3649-4816-b9d3-c7c1fee327cf admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:28.476906 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ae450-b3e3-4498-aa41-7f16676570b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:28.477625 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ae450-b3e3-4498-aa41-7f16676570b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:28.477914 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ae450-b3e3-4498-aa41-7f16676570b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:28.478185 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ae450-b3e3-4498-aa41-7f16676570b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:28.478872 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ae450-b3e3-4498-aa41-7f16676570b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:28.480497 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ae450-b3e3-4498-aa41-7f16676570b5 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:28.493731 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1374/2735] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:28.504072 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc6be8a-0dcd-4428-85ab-a05b5af74b01 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:28.514339 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc6be8a-0dcd-4428-85ab-a05b5af74b01 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:28.519489 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac8906-3cfe-4180-92b0-16560a3d6a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:28.520148 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac8906-3cfe-4180-92b0-16560a3d6a29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:28.520428 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac8906-3cfe-4180-92b0-16560a3d6a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:28.520727 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac8906-3cfe-4180-92b0-16560a3d6a29 admin admin] PATH_INFO: `/v3/projects/d057f29b771a4110be66186949246663/users/7338eed24b4a42208d4395e93d464d65/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:28.538252 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ac8906-3cfe-4180-92b0-16560a3d6a29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d057f29b771a4110be66186949246663, user_id=7338eed24b4a42208d4395e93d464d65, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:28.541013 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ac8906-3cfe-4180-92b0-16560a3d6a29 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:28.633772 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1362/2736] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:04:28 2022] PUT /identity/v3/projects/d057f29b771a4110be66186949246663/users/7338eed24b4a42208d4395e93d464d65/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:04:28.638963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c447ecb-01c3-42f1-92f4-29966c1ab118 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:28.639249 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c447ecb-01c3-42f1-92f4-29966c1ab118 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:28.639565 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c447ecb-01c3-42f1-92f4-29966c1ab118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:28.659628 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3c447ecb-01c3-42f1-92f4-29966c1ab118 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:28.660855 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3c447ecb-01c3-42f1-92f4-29966c1ab118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:28.702453 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:28.721293 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1375/2737] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:28.730601 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac8906-3cfe-4180-92b0-16560a3d6a29 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:28.732398 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-31ac8906-3cfe-4180-92b0-16560a3d6a29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:28.788583 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac8906-3cfe-4180-92b0-16560a3d6a29 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:28.792196 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b4475-7366-482a-b38f-ce7332ea5a2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:28.792817 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b4475-7366-482a-b38f-ce7332ea5a2b service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:28.793098 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b4475-7366-482a-b38f-ce7332ea5a2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:28.793371 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b4475-7366-482a-b38f-ce7332ea5a2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:28.802965 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557b4475-7366-482a-b38f-ce7332ea5a2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:28.804415 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557b4475-7366-482a-b38f-ce7332ea5a2b service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:28.813660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1363/2738] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:04:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:04:34.500057 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37371089-9f42-4a24-9104-4a3dcd390b89 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:34.500920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37371089-9f42-4a24-9104-4a3dcd390b89 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:34.500920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37371089-9f42-4a24-9104-4a3dcd390b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:34.538932 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-37371089-9f42-4a24-9104-4a3dcd390b89 None None] MFA Rules not processed for user `7338eed24b4a42208d4395e93d464d65`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:34.540189 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-37371089-9f42-4a24-9104-4a3dcd390b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:34.612830 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1376/2739] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:04:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:04:34.617428 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b4475-7366-482a-b38f-ce7332ea5a2b service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:34.628485 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b4475-7366-482a-b38f-ce7332ea5a2b service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:34.633352 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fc945-383c-43e4-97d0-43061e265ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:34.633979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fc945-383c-43e4-97d0-43061e265ab9 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:34.634261 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fc945-383c-43e4-97d0-43061e265ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:34.634534 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fc945-383c-43e4-97d0-43061e265ab9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:34.635203 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353fc945-383c-43e4-97d0-43061e265ab9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:34.637158 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353fc945-383c-43e4-97d0-43061e265ab9 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:34.729560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1364/2740] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 15:04:34 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:04:34.733604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37371089-9f42-4a24-9104-4a3dcd390b89 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:34.745615 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37371089-9f42-4a24-9104-4a3dcd390b89 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:34.818047 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba0c48-c77d-427f-bfa0-4bc697eac321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:34.819127 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba0c48-c77d-427f-bfa0-4bc697eac321 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:34.819681 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba0c48-c77d-427f-bfa0-4bc697eac321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:34.820049 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba0c48-c77d-427f-bfa0-4bc697eac321 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:34.820794 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ba0c48-c77d-427f-bfa0-4bc697eac321 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:34.823007 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ba0c48-c77d-427f-bfa0-4bc697eac321 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:34.944163 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1377/2741] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 15:04:34 2022] POST /identity/v3/users => generated 299 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:04:34.948228 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fc945-383c-43e4-97d0-43061e265ab9 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:34.959718 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fc945-383c-43e4-97d0-43061e265ab9 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:34.965007 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aafe7e0-e065-411b-8872-a989697ae6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:34.965715 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aafe7e0-e065-411b-8872-a989697ae6a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:34.966070 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aafe7e0-e065-411b-8872-a989697ae6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:34.966442 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aafe7e0-e065-411b-8872-a989697ae6a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:34.967217 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aafe7e0-e065-411b-8872-a989697ae6a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:34.969152 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aafe7e0-e065-411b-8872-a989697ae6a5 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:34.983843 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1365/2742] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:34.985394 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba0c48-c77d-427f-bfa0-4bc697eac321 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:34.996090 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba0c48-c77d-427f-bfa0-4bc697eac321 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:35.000997 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea67f8f-2d79-438f-ac98-218a006724ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:35.001848 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea67f8f-2d79-438f-ac98-218a006724ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:35.002193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea67f8f-2d79-438f-ac98-218a006724ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:35.002540 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea67f8f-2d79-438f-ac98-218a006724ca admin admin] PATH_INFO: `/v3/projects/d1999761b68543bdb13a292635ca9e56/users/9f6c250778da4199887ac3319244d226/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:35.020054 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea67f8f-2d79-438f-ac98-218a006724ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1999761b68543bdb13a292635ca9e56, user_id=9f6c250778da4199887ac3319244d226, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:35.023533 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea67f8f-2d79-438f-ac98-218a006724ca admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:35.125407 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1378/2743] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:04:34 2022] PUT /identity/v3/projects/d1999761b68543bdb13a292635ca9e56/users/9f6c250778da4199887ac3319244d226/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:04:35.130322 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aafe7e0-e065-411b-8872-a989697ae6a5 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:35.144839 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aafe7e0-e065-411b-8872-a989697ae6a5 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:35.206400 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe62bf-bf2a-4526-a53f-654b3327beab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:35.206940 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe62bf-bf2a-4526-a53f-654b3327beab admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:35.207223 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe62bf-bf2a-4526-a53f-654b3327beab admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:35.207535 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe62bf-bf2a-4526-a53f-654b3327beab admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:35.208224 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fe62bf-bf2a-4526-a53f-654b3327beab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:35.209803 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fe62bf-bf2a-4526-a53f-654b3327beab admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:35.219853 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1366/2744] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:35.224268 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea67f8f-2d79-438f-ac98-218a006724ca admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:35.234927 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea67f8f-2d79-438f-ac98-218a006724ca admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:35.240554 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27edef85-e843-44ec-a44b-620241e6bafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:35.241160 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27edef85-e843-44ec-a44b-620241e6bafa admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:35.241345 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27edef85-e843-44ec-a44b-620241e6bafa admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:35.241627 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27edef85-e843-44ec-a44b-620241e6bafa admin admin] PATH_INFO: `/v3/projects/d1999761b68543bdb13a292635ca9e56/users/9f6c250778da4199887ac3319244d226/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:35.243656 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27edef85-e843-44ec-a44b-620241e6bafa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1999761b68543bdb13a292635ca9e56, user_id=9f6c250778da4199887ac3319244d226, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:35.246336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27edef85-e843-44ec-a44b-620241e6bafa admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:35.257983 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1379/2745] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:04:35 2022] PUT /identity/v3/projects/d1999761b68543bdb13a292635ca9e56/users/9f6c250778da4199887ac3319244d226/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:04:41.998375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02eaca-eb40-4bc4-ba75-3f66ea39de59 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:41.998975 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02eaca-eb40-4bc4-ba75-3f66ea39de59 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:41.998975 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02eaca-eb40-4bc4-ba75-3f66ea39de59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:42.029895 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ef02eaca-eb40-4bc4-ba75-3f66ea39de59 None None] MFA Rules not processed for user `9f6c250778da4199887ac3319244d226`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:42.031266 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ef02eaca-eb40-4bc4-ba75-3f66ea39de59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:42.067979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:42.115029 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1367/2746] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:04:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:42.127054 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27edef85-e843-44ec-a44b-620241e6bafa admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:42.128999 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-27edef85-e843-44ec-a44b-620241e6bafa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:42.163680 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:42.218974 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27edef85-e843-44ec-a44b-620241e6bafa admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:42.223487 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e984a-b8d6-404b-b0af-00c99589737c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62adb1f5e9654927847e1717630ff6b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:42.224137 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e984a-b8d6-404b-b0af-00c99589737c service glance] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:42.224419 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e984a-b8d6-404b-b0af-00c99589737c service glance] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:42.224697 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e984a-b8d6-404b-b0af-00c99589737c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:42.321463 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e984a-b8d6-404b-b0af-00c99589737c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:42.322924 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e984a-b8d6-404b-b0af-00c99589737c service glance] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:42.337151 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1380/2747] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:04:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:04:42.619277 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc4397-7691-4372-88b6-9675d5962821 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:42.619779 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc4397-7691-4372-88b6-9675d5962821 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:42.620149 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc4397-7691-4372-88b6-9675d5962821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:42.645994 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b9bc4397-7691-4372-88b6-9675d5962821 None None] MFA Rules not processed for user `5179c3c4eaaa41f3beb79ba7f1993478`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:42.647472 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b9bc4397-7691-4372-88b6-9675d5962821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:42.716394 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1368/2748] 10.222.0.97 () {58 vars in 1009 bytes} [Tue Jul 26 15:04:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:42.739783 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e984a-b8d6-404b-b0af-00c99589737c service glance] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:42.741284 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f65e984a-b8d6-404b-b0af-00c99589737c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:42.819229 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e984a-b8d6-404b-b0af-00c99589737c service glance] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:42.825107 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe329b-b130-4310-9dd3-042fb066d68a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85122c4052fd4ba19a2dae185a707caf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:42.825756 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe329b-b130-4310-9dd3-042fb066d68a service swift] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:42.826037 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe329b-b130-4310-9dd3-042fb066d68a service swift] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:42.826492 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe329b-b130-4310-9dd3-042fb066d68a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:42.837468 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe329b-b130-4310-9dd3-042fb066d68a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:42.839148 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe329b-b130-4310-9dd3-042fb066d68a service swift] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:42.857566 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1381/2749] 10.222.0.97 () {60 vars in 1384 bytes} [Tue Jul 26 15:04:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:04:50.266926 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f03b896-b766-4a15-bb29-900e0f3f6a29 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:50.267534 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f03b896-b766-4a15-bb29-900e0f3f6a29 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:50.267534 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f03b896-b766-4a15-bb29-900e0f3f6a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:50.289015 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-8f03b896-b766-4a15-bb29-900e0f3f6a29 None None] MFA Rules not processed for user `df675a52f9b245198897842f7a6ecd13`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:50.290264 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8f03b896-b766-4a15-bb29-900e0f3f6a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:50.368878 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1369/2750] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:04:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:50.779052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe329b-b130-4310-9dd3-042fb066d68a service swift] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:50.863946 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe329b-b130-4310-9dd3-042fb066d68a service swift] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:50.869526 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe4fdf-c01a-4b01-9c7b-71d7bf84d6c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:50.870854 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe4fdf-c01a-4b01-9c7b-71d7bf84d6c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:50.871332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe4fdf-c01a-4b01-9c7b-71d7bf84d6c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:50.871846 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe4fdf-c01a-4b01-9c7b-71d7bf84d6c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:50.874860 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2cbe4fdf-c01a-4b01-9c7b-71d7bf84d6c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:50.881980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:50.888250 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe4fdf-c01a-4b01-9c7b-71d7bf84d6c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:50.889964 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe4fdf-c01a-4b01-9c7b-71d7bf84d6c5 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:50.899660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1382/2751] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:04:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:04:51.410759 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f03b896-b766-4a15-bb29-900e0f3f6a29 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:51.412292 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8f03b896-b766-4a15-bb29-900e0f3f6a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:51.512106 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f03b896-b766-4a15-bb29-900e0f3f6a29 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:51.521101 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03073dd1-2514-4076-9e73-986854b764d9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '252c2d2939134416a3e8efb254f03c5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:51.522005 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03073dd1-2514-4076-9e73-986854b764d9 service placement] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:51.522289 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03073dd1-2514-4076-9e73-986854b764d9 service placement] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:51.522569 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03073dd1-2514-4076-9e73-986854b764d9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:51.524478 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-03073dd1-2514-4076-9e73-986854b764d9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:51.539165 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03073dd1-2514-4076-9e73-986854b764d9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:51.540840 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03073dd1-2514-4076-9e73-986854b764d9 service placement] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:51.554295 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1370/2752] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:04:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:04:54.133246 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b6377-217b-42e1-80e4-0a04eaaf2324 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:54.133818 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b6377-217b-42e1-80e4-0a04eaaf2324 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:54.133818 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b6377-217b-42e1-80e4-0a04eaaf2324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:54.153837 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-601b6377-217b-42e1-80e4-0a04eaaf2324 None None] MFA Rules not processed for user `7338eed24b4a42208d4395e93d464d65`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:54.155045 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-601b6377-217b-42e1-80e4-0a04eaaf2324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:54.162732 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1383/2753] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:04:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:54.421862 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03073dd1-2514-4076-9e73-986854b764d9 service placement] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:54.423281 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-03073dd1-2514-4076-9e73-986854b764d9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:54.432923 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:54.438375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03073dd1-2514-4076-9e73-986854b764d9 service placement] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:54.442141 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d23b2e-caba-4948-81bf-f34ae495fbe7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:54.442762 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d23b2e-caba-4948-81bf-f34ae495fbe7 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:54.443041 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d23b2e-caba-4948-81bf-f34ae495fbe7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:54.443314 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d23b2e-caba-4948-81bf-f34ae495fbe7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:54.445177 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-17d23b2e-caba-4948-81bf-f34ae495fbe7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:54.456104 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d23b2e-caba-4948-81bf-f34ae495fbe7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:54.457576 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d23b2e-caba-4948-81bf-f34ae495fbe7 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:54.466477 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1371/2754] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:04:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:05:01.076699 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930da637-fa01-456a-8ff0-7f1f17c1de65 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:01.077188 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930da637-fa01-456a-8ff0-7f1f17c1de65 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:01.077283 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930da637-fa01-456a-8ff0-7f1f17c1de65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:01.096889 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-930da637-fa01-456a-8ff0-7f1f17c1de65 None None] MFA Rules not processed for user `5179c3c4eaaa41f3beb79ba7f1993478`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:01.098084 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-930da637-fa01-456a-8ff0-7f1f17c1de65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:01.105604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1384/2755] 10.222.0.97 () {58 vars in 1009 bytes} [Tue Jul 26 15:05:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:01.117873 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d23b2e-caba-4948-81bf-f34ae495fbe7 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:01.130757 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d23b2e-caba-4948-81bf-f34ae495fbe7 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:01.135763 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4ec88-0e7f-4805-bc63-f7443a78384e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85122c4052fd4ba19a2dae185a707caf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:01.136380 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4ec88-0e7f-4805-bc63-f7443a78384e service swift] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:01.136665 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4ec88-0e7f-4805-bc63-f7443a78384e service swift] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:01.136943 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4ec88-0e7f-4805-bc63-f7443a78384e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:01.146551 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b4ec88-0e7f-4805-bc63-f7443a78384e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:01.148244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b4ec88-0e7f-4805-bc63-f7443a78384e service swift] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:01.156759 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1372/2756] 10.222.0.97 () {60 vars in 1384 bytes} [Tue Jul 26 15:05:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:05:04.592789 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930da637-fa01-456a-8ff0-7f1f17c1de65 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:04.594784 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-930da637-fa01-456a-8ff0-7f1f17c1de65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:04.677680 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930da637-fa01-456a-8ff0-7f1f17c1de65 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:04.683273 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fb0be-89d8-400b-8b9f-e3f6d532d61c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:04.683930 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fb0be-89d8-400b-8b9f-e3f6d532d61c service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:04.684220 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fb0be-89d8-400b-8b9f-e3f6d532d61c service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:04.684493 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fb0be-89d8-400b-8b9f-e3f6d532d61c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:04.686294 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1f8fb0be-89d8-400b-8b9f-e3f6d532d61c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:04.699630 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8fb0be-89d8-400b-8b9f-e3f6d532d61c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:04.701344 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8fb0be-89d8-400b-8b9f-e3f6d532d61c service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:04.716949 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1385/2757] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:05:11.153350 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=10 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:12.714130 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4ec88-0e7f-4805-bc63-f7443a78384e service swift] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:12.734285 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4ec88-0e7f-4805-bc63-f7443a78384e service swift] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:12.740744 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b106f944-f197-41c4-bafc-bdc71e82f3ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:12.741438 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106f944-f197-41c4-bafc-bdc71e82f3ce service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:12.741742 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106f944-f197-41c4-bafc-bdc71e82f3ce service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:12.742045 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106f944-f197-41c4-bafc-bdc71e82f3ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:12.744038 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b106f944-f197-41c4-bafc-bdc71e82f3ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:12.757892 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b106f944-f197-41c4-bafc-bdc71e82f3ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:12.759716 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b106f944-f197-41c4-bafc-bdc71e82f3ce service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:12.773541 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1373/2758] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:05:14.709853 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=43 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:18.120207 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fb0be-89d8-400b-8b9f-e3f6d532d61c service nova] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:18.134166 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fb0be-89d8-400b-8b9f-e3f6d532d61c service nova] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:18.140555 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f931349d-780d-4fed-b98f-658597df8496 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:18.141193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931349d-780d-4fed-b98f-658597df8496 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:18.141479 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931349d-780d-4fed-b98f-658597df8496 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:18.141757 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931349d-780d-4fed-b98f-658597df8496 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:18.143648 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f931349d-780d-4fed-b98f-658597df8496 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:18.159256 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f931349d-780d-4fed-b98f-658597df8496 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:18.160891 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f931349d-780d-4fed-b98f-658597df8496 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:18.172189 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1386/2759] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:05:22.766311 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=6 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:28.167261 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=6 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:39.593994 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b106f944-f197-41c4-bafc-bdc71e82f3ce service nova] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:39.603664 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b106f944-f197-41c4-bafc-bdc71e82f3ce service nova] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:39.607270 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8fab9-8c13-4ee7-98c6-504debab3225 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:39.608436 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8fab9-8c13-4ee7-98c6-504debab3225 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:39.608722 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8fab9-8c13-4ee7-98c6-504debab3225 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:39.608995 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8fab9-8c13-4ee7-98c6-504debab3225 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:39.618134 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d8fab9-8c13-4ee7-98c6-504debab3225 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:39.619662 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d8fab9-8c13-4ee7-98c6-504debab3225 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:39.630002 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1374/2760] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:05:49.625731 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=6 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:52.978887 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f931349d-780d-4fed-b98f-658597df8496 service nova] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:52.990980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f931349d-780d-4fed-b98f-658597df8496 service nova] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:52.996095 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e8880-ee6a-4cbc-8081-b527c2e42b0f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '252c2d2939134416a3e8efb254f03c5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:52.996714 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e8880-ee6a-4cbc-8081-b527c2e42b0f service placement] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:52.997000 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e8880-ee6a-4cbc-8081-b527c2e42b0f service placement] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:52.997287 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e8880-ee6a-4cbc-8081-b527c2e42b0f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:52.999129 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-af2e8880-ee6a-4cbc-8081-b527c2e42b0f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:53.012780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e8880-ee6a-4cbc-8081-b527c2e42b0f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:53.014385 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e8880-ee6a-4cbc-8081-b527c2e42b0f service placement] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:53.024522 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1387/2761] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:06:03.020041 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=6 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:50.620664 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8fab9-8c13-4ee7-98c6-504debab3225 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:50.629801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8fab9-8c13-4ee7-98c6-504debab3225 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:50.633460 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c698f-441b-4783-9f28-343645434033 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:50.634063 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c698f-441b-4783-9f28-343645434033 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:50.634346 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c698f-441b-4783-9f28-343645434033 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:50.634620 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c698f-441b-4783-9f28-343645434033 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:50.636930 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-002c698f-441b-4783-9f28-343645434033 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:50.651446 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002c698f-441b-4783-9f28-343645434033 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:50.652868 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002c698f-441b-4783-9f28-343645434033 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:50.663897 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1375/2762] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:06:55.459144 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc0961-9411-447e-a4db-021f6e9b3985 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:55.459810 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc0961-9411-447e-a4db-021f6e9b3985 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:55.459810 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc0961-9411-447e-a4db-021f6e9b3985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:55.490721 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-45cc0961-9411-447e-a4db-021f6e9b3985 None None] MFA Rules not processed for user `5179c3c4eaaa41f3beb79ba7f1993478`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:55.492016 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-45cc0961-9411-447e-a4db-021f6e9b3985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:55.501689 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1388/2763] 10.222.0.97 () {58 vars in 1009 bytes} [Tue Jul 26 15:06:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:55.558192 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c698f-441b-4783-9f28-343645434033 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:55.579197 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c698f-441b-4783-9f28-343645434033 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:55.586043 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe57613e-6872-4b65-a4ba-316827e331f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85122c4052fd4ba19a2dae185a707caf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:55.586804 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57613e-6872-4b65-a4ba-316827e331f0 service swift] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:55.587213 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57613e-6872-4b65-a4ba-316827e331f0 service swift] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:55.588303 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57613e-6872-4b65-a4ba-316827e331f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:55.600876 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe57613e-6872-4b65-a4ba-316827e331f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:55.602718 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe57613e-6872-4b65-a4ba-316827e331f0 service swift] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:55.612595 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1376/2764] 10.222.0.97 () {60 vars in 1384 bytes} [Tue Jul 26 15:06:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:07:02.466004 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc0961-9411-447e-a4db-021f6e9b3985 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:02.476739 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc0961-9411-447e-a4db-021f6e9b3985 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:02.483952 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bab357-eee6-4e36-8e4d-61241038e27d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:02.484661 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bab357-eee6-4e36-8e4d-61241038e27d service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:02.485198 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bab357-eee6-4e36-8e4d-61241038e27d service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:02.485709 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bab357-eee6-4e36-8e4d-61241038e27d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:02.488005 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-57bab357-eee6-4e36-8e4d-61241038e27d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:02.509683 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bab357-eee6-4e36-8e4d-61241038e27d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:02.511963 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bab357-eee6-4e36-8e4d-61241038e27d service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:02.527920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1389/2765] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:07:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:07:05.610491 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=12 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:12.518550 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=9 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:14.976025 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718d952-d3ee-4570-aeaa-466ed77c70b0 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:14.976680 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718d952-d3ee-4570-aeaa-466ed77c70b0 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:14.976680 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718d952-d3ee-4570-aeaa-466ed77c70b0 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:14.978167 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1377/2766] 10.222.0.97 () {54 vars in 921 bytes} [Tue Jul 26 15:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 15:07:14.983613 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29de74-ae56-4889-86f6-d36cd410d24e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:14.984119 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29de74-ae56-4889-86f6-d36cd410d24e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:14.984235 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29de74-ae56-4889-86f6-d36cd410d24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:15.007532 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-fb29de74-ae56-4889-86f6-d36cd410d24e None None] MFA Rules not processed for user `266abef6cda5495b897170f92043aa03`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:15.009448 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fb29de74-ae56-4889-86f6-d36cd410d24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:15.019388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1390/2767] 10.222.0.97 () {58 vars in 1018 bytes} [Tue Jul 26 15:07:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:15.032610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718d952-d3ee-4570-aeaa-466ed77c70b0 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:15.045408 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718d952-d3ee-4570-aeaa-466ed77c70b0 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:15.056117 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd107f8-b903-4359-aab0-f53fdb87f1c9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62adb1f5e9654927847e1717630ff6b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:15.056798 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd107f8-b903-4359-aab0-f53fdb87f1c9 service glance] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:15.057085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd107f8-b903-4359-aab0-f53fdb87f1c9 service glance] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:15.057360 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd107f8-b903-4359-aab0-f53fdb87f1c9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:15.068260 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd107f8-b903-4359-aab0-f53fdb87f1c9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:15.069686 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd107f8-b903-4359-aab0-f53fdb87f1c9 service glance] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:15.082053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1378/2768] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:07:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:07:15.365935 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29de74-ae56-4889-86f6-d36cd410d24e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:15.375844 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29de74-ae56-4889-86f6-d36cd410d24e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:15.380167 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524e007-9748-45fb-bf9e-7185a237e7fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:15.380801 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524e007-9748-45fb-bf9e-7185a237e7fd service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:15.381596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524e007-9748-45fb-bf9e-7185a237e7fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:15.381888 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524e007-9748-45fb-bf9e-7185a237e7fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:15.384825 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9524e007-9748-45fb-bf9e-7185a237e7fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:15.397739 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9524e007-9748-45fb-bf9e-7185a237e7fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:15.399260 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9524e007-9748-45fb-bf9e-7185a237e7fd service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:15.408486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1391/2769] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:07:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:07:15.485896 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd107f8-b903-4359-aab0-f53fdb87f1c9 service glance] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:15.495589 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd107f8-b903-4359-aab0-f53fdb87f1c9 service glance] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:15.499661 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2b7a0-6bc8-4139-8809-caa868395834 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:15.500287 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2b7a0-6bc8-4139-8809-caa868395834 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:15.500573 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2b7a0-6bc8-4139-8809-caa868395834 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:15.500852 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2b7a0-6bc8-4139-8809-caa868395834 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:15.503782 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1cd2b7a0-6bc8-4139-8809-caa868395834 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:15.519963 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2b7a0-6bc8-4139-8809-caa868395834 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:15.521425 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2b7a0-6bc8-4139-8809-caa868395834 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:15.534227 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1379/2770] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:07:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:07:18.374085 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95278d-ed3d-422c-85b6-5e4e349ec84d None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:18.374778 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95278d-ed3d-422c-85b6-5e4e349ec84d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:18.374778 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95278d-ed3d-422c-85b6-5e4e349ec84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:18.397665 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ea95278d-ed3d-422c-85b6-5e4e349ec84d None None] MFA Rules not processed for user `5179c3c4eaaa41f3beb79ba7f1993478`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:18.399759 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ea95278d-ed3d-422c-85b6-5e4e349ec84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:18.410294 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1392/2771] 10.222.0.97 () {58 vars in 1009 bytes} [Tue Jul 26 15:07:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:07:18.419957 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2b7a0-6bc8-4139-8809-caa868395834 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:18.434085 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2b7a0-6bc8-4139-8809-caa868395834 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:18.438308 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061537e3-871a-4a94-90d4-7254cd0093a8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85122c4052fd4ba19a2dae185a707caf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:18.438995 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061537e3-871a-4a94-90d4-7254cd0093a8 service swift] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:18.439239 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061537e3-871a-4a94-90d4-7254cd0093a8 service swift] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:18.439562 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061537e3-871a-4a94-90d4-7254cd0093a8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:18.452065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061537e3-871a-4a94-90d4-7254cd0093a8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:18.453771 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061537e3-871a-4a94-90d4-7254cd0093a8 service swift] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:18.468822 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1380/2772] 10.222.0.97 () {60 vars in 1384 bytes} [Tue Jul 26 15:07:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:07:28.390745 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=12 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:28.459724 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=18 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:37.933053 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682fd6aa-52b6-4e4c-84dd-bfaece754755 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:37.933530 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682fd6aa-52b6-4e4c-84dd-bfaece754755 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:37.933648 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682fd6aa-52b6-4e4c-84dd-bfaece754755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:37.954867 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-682fd6aa-52b6-4e4c-84dd-bfaece754755 None None] MFA Rules not processed for user `5179c3c4eaaa41f3beb79ba7f1993478`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:37.956114 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-682fd6aa-52b6-4e4c-84dd-bfaece754755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:37.964061 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1393/2773] 10.222.0.97 () {58 vars in 967 bytes} [Tue Jul 26 15:07:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:37.976597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061537e3-871a-4a94-90d4-7254cd0093a8 service swift] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:37.987293 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061537e3-871a-4a94-90d4-7254cd0093a8 service swift] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:37.992444 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4537b5-fed5-4cce-ad6f-db2984f8a39f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85122c4052fd4ba19a2dae185a707caf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:37.993057 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4537b5-fed5-4cce-ad6f-db2984f8a39f service swift] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:37.993368 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4537b5-fed5-4cce-ad6f-db2984f8a39f service swift] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:37.993646 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4537b5-fed5-4cce-ad6f-db2984f8a39f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:38.002031 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4537b5-fed5-4cce-ad6f-db2984f8a39f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:38.003648 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4537b5-fed5-4cce-ad6f-db2984f8a39f service swift] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:38.012276 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1381/2774] 10.222.0.97 () {60 vars in 1384 bytes} [Tue Jul 26 15:07:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:07:47.948168 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:48.008990 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=6 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:59.668494 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682fd6aa-52b6-4e4c-84dd-bfaece754755 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:59.677696 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682fd6aa-52b6-4e4c-84dd-bfaece754755 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:59.681277 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949103a7-659e-4b3d-9cb8-4ab065d537d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:59.681884 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949103a7-659e-4b3d-9cb8-4ab065d537d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:59.682163 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949103a7-659e-4b3d-9cb8-4ab065d537d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:59.682434 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949103a7-659e-4b3d-9cb8-4ab065d537d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:59.684203 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-949103a7-659e-4b3d-9cb8-4ab065d537d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:59.694691 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949103a7-659e-4b3d-9cb8-4ab065d537d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:59.696321 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949103a7-659e-4b3d-9cb8-4ab065d537d8 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:59.704982 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1394/2775] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:07:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:08:09.701200 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=6 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:08:15.542583 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc5614-8aa8-496e-ad29-5e621adf7853 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:15.543125 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc5614-8aa8-496e-ad29-5e621adf7853 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:15.543226 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc5614-8aa8-496e-ad29-5e621adf7853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:15.563428 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6dfc5614-8aa8-496e-ad29-5e621adf7853 None None] MFA Rules not processed for user `5179c3c4eaaa41f3beb79ba7f1993478`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:08:15.564571 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6dfc5614-8aa8-496e-ad29-5e621adf7853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:15.572355 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1382/2776] 10.222.0.97 () {58 vars in 967 bytes} [Tue Jul 26 15:08:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:08:15.595038 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949103a7-659e-4b3d-9cb8-4ab065d537d8 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:15.607256 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949103a7-659e-4b3d-9cb8-4ab065d537d8 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:15.612236 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930840e-c269-450b-857e-ef3a94ab065c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85122c4052fd4ba19a2dae185a707caf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:15.612839 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930840e-c269-450b-857e-ef3a94ab065c service swift] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:15.613121 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930840e-c269-450b-857e-ef3a94ab065c service swift] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:15.613399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930840e-c269-450b-857e-ef3a94ab065c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:15.621504 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930840e-c269-450b-857e-ef3a94ab065c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:15.623125 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930840e-c269-450b-857e-ef3a94ab065c service swift] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:15.632089 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1395/2777] 10.222.0.97 () {60 vars in 1384 bytes} [Tue Jul 26 15:08:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:08:24.851565 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc5614-8aa8-496e-ad29-5e621adf7853 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:24.863415 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6dfc5614-8aa8-496e-ad29-5e621adf7853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:24.868656 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc5614-8aa8-496e-ad29-5e621adf7853 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:24.873905 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7edb14-a463-4e4d-9c6e-ad72801bd99a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '252c2d2939134416a3e8efb254f03c5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:24.874611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7edb14-a463-4e4d-9c6e-ad72801bd99a service placement] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:24.874894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7edb14-a463-4e4d-9c6e-ad72801bd99a service placement] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:24.875166 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7edb14-a463-4e4d-9c6e-ad72801bd99a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:24.876989 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ee7edb14-a463-4e4d-9c6e-ad72801bd99a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:24.890200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7edb14-a463-4e4d-9c6e-ad72801bd99a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:24.891968 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7edb14-a463-4e4d-9c6e-ad72801bd99a service placement] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:24.903704 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1383/2778] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:08:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:08:25.116394 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930840e-c269-450b-857e-ef3a94ab065c service swift] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:25.182054 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930840e-c269-450b-857e-ef3a94ab065c service swift] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:25.186567 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b5730e-f932-43b1-a89e-075dbbf46207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:25.187211 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5730e-f932-43b1-a89e-075dbbf46207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:25.187531 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5730e-f932-43b1-a89e-075dbbf46207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:25.187811 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5730e-f932-43b1-a89e-075dbbf46207 admin admin] PATH_INFO: `/v3/users/7338eed24b4a42208d4395e93d464d65` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:25.189058 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b5730e-f932-43b1-a89e-075dbbf46207 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7338eed24b4a42208d4395e93d464d65)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:25.191149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b5730e-f932-43b1-a89e-075dbbf46207 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:25.496157 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d8b5730e-f932-43b1-a89e-075dbbf46207 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7338eed24b4a42208d4395e93d464d65'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:25.496748 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d8b5730e-f932-43b1-a89e-075dbbf46207 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7338eed24b4a42208d4395e93d464d65'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:25.512499 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d8b5730e-f932-43b1-a89e-075dbbf46207 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7338eed24b4a42208d4395e93d464d65'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:25.519676 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d8b5730e-f932-43b1-a89e-075dbbf46207 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7338eed24b4a42208d4395e93d464d65'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:25.520625 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d8b5730e-f932-43b1-a89e-075dbbf46207 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7338eed24b4a42208d4395e93d464d65'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:25.624668 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1396/2779] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:08:25 2022] DELETE /identity/v3/users/7338eed24b4a42208d4395e93d464d65 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:08:25.884284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7edb14-a463-4e4d-9c6e-ad72801bd99a service placement] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:25.896521 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7edb14-a463-4e4d-9c6e-ad72801bd99a service placement] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:25.959650 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e65be5-2ec8-4791-8678-0eccebe1af1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:25.960301 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e65be5-2ec8-4791-8678-0eccebe1af1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:25.960583 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e65be5-2ec8-4791-8678-0eccebe1af1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:25.960858 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e65be5-2ec8-4791-8678-0eccebe1af1e admin admin] PATH_INFO: `/v3/projects/d057f29b771a4110be66186949246663` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:25.962030 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e65be5-2ec8-4791-8678-0eccebe1af1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d057f29b771a4110be66186949246663)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:25.963993 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e65be5-2ec8-4791-8678-0eccebe1af1e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:26.109047 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-06e65be5-2ec8-4791-8678-0eccebe1af1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd057f29b771a4110be66186949246663', 'request_id': 'req-06e65be5-2ec8-4791-8678-0eccebe1af1e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:26.115307 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-06e65be5-2ec8-4791-8678-0eccebe1af1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd057f29b771a4110be66186949246663', 'request_id': 'req-06e65be5-2ec8-4791-8678-0eccebe1af1e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:26.242430 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-06e65be5-2ec8-4791-8678-0eccebe1af1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd057f29b771a4110be66186949246663', 'request_id': 'req-06e65be5-2ec8-4791-8678-0eccebe1af1e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:26.254635 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-06e65be5-2ec8-4791-8678-0eccebe1af1e admin admin] The token cache is being invalidate because project d057f29b771a4110be66186949246663 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:08:26.254930 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-06e65be5-2ec8-4791-8678-0eccebe1af1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:26.255713 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-06e65be5-2ec8-4791-8678-0eccebe1af1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:26.257324 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1384/2780] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:08:25 2022] DELETE /identity/v3/projects/d057f29b771a4110be66186949246663 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:08:26.261485 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b5730e-f932-43b1-a89e-075dbbf46207 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:26.262943 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d8b5730e-f932-43b1-a89e-075dbbf46207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:26.277032 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b5730e-f932-43b1-a89e-075dbbf46207 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:26.345629 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5885b2a7-8f9d-4ccf-85d0-4c7c0fb80b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:26.346212 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885b2a7-8f9d-4ccf-85d0-4c7c0fb80b18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:26.346500 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885b2a7-8f9d-4ccf-85d0-4c7c0fb80b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:26.346778 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885b2a7-8f9d-4ccf-85d0-4c7c0fb80b18 admin admin] PATH_INFO: `/v3/users/9f6c250778da4199887ac3319244d226` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:26.348013 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5885b2a7-8f9d-4ccf-85d0-4c7c0fb80b18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f6c250778da4199887ac3319244d226)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:26.350527 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5885b2a7-8f9d-4ccf-85d0-4c7c0fb80b18 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:26.356954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:08:26.581297 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5885b2a7-8f9d-4ccf-85d0-4c7c0fb80b18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f6c250778da4199887ac3319244d226'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:26.582331 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5885b2a7-8f9d-4ccf-85d0-4c7c0fb80b18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f6c250778da4199887ac3319244d226'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:26.596405 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5885b2a7-8f9d-4ccf-85d0-4c7c0fb80b18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f6c250778da4199887ac3319244d226'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:26.604493 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5885b2a7-8f9d-4ccf-85d0-4c7c0fb80b18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f6c250778da4199887ac3319244d226'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:26.605267 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5885b2a7-8f9d-4ccf-85d0-4c7c0fb80b18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f6c250778da4199887ac3319244d226'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:26.696600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1397/2781] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:08:26 2022] DELETE /identity/v3/users/9f6c250778da4199887ac3319244d226 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:08:26.964080 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e65be5-2ec8-4791-8678-0eccebe1af1e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:26.965647 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-06e65be5-2ec8-4791-8678-0eccebe1af1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:26.979927 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e65be5-2ec8-4791-8678-0eccebe1af1e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:27.045479 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e9105-3c5c-4a7b-b0c7-ea2f70d1327f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:27.046137 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e9105-3c5c-4a7b-b0c7-ea2f70d1327f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:27.046413 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e9105-3c5c-4a7b-b0c7-ea2f70d1327f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:27.046771 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e9105-3c5c-4a7b-b0c7-ea2f70d1327f admin admin] PATH_INFO: `/v3/projects/d1999761b68543bdb13a292635ca9e56` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:27.047908 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e9105-3c5c-4a7b-b0c7-ea2f70d1327f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1999761b68543bdb13a292635ca9e56)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:27.049965 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e9105-3c5c-4a7b-b0c7-ea2f70d1327f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:27.062242 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:08:27.183336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-7d0e9105-3c5c-4a7b-b0c7-ea2f70d1327f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1999761b68543bdb13a292635ca9e56', 'request_id': 'req-7d0e9105-3c5c-4a7b-b0c7-ea2f70d1327f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:27.188841 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-7d0e9105-3c5c-4a7b-b0c7-ea2f70d1327f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1999761b68543bdb13a292635ca9e56', 'request_id': 'req-7d0e9105-3c5c-4a7b-b0c7-ea2f70d1327f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:27.267724 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-7d0e9105-3c5c-4a7b-b0c7-ea2f70d1327f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1999761b68543bdb13a292635ca9e56', 'request_id': 'req-7d0e9105-3c5c-4a7b-b0c7-ea2f70d1327f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:27.280905 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-7d0e9105-3c5c-4a7b-b0c7-ea2f70d1327f admin admin] The token cache is being invalidate because project d1999761b68543bdb13a292635ca9e56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:08:27.281477 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-7d0e9105-3c5c-4a7b-b0c7-ea2f70d1327f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:27.282748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-7d0e9105-3c5c-4a7b-b0c7-ea2f70d1327f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:27.284376 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1385/2782] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:08:26 2022] DELETE /identity/v3/projects/d1999761b68543bdb13a292635ca9e56 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:08:36.693393 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:08:37.413117 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=19,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:08:39.295286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5885b2a7-8f9d-4ccf-85d0-4c7c0fb80b18 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:39.297573 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5885b2a7-8f9d-4ccf-85d0-4c7c0fb80b18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:39.378680 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5885b2a7-8f9d-4ccf-85d0-4c7c0fb80b18 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:39.447626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2530428-aa7c-4ffc-a019-107df6b5e873 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:39.448256 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2530428-aa7c-4ffc-a019-107df6b5e873 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:39.448541 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2530428-aa7c-4ffc-a019-107df6b5e873 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:39.448825 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2530428-aa7c-4ffc-a019-107df6b5e873 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:39.450663 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e2530428-aa7c-4ffc-a019-107df6b5e873 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:39.464258 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2530428-aa7c-4ffc-a019-107df6b5e873 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:39.465860 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2530428-aa7c-4ffc-a019-107df6b5e873 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:39.478310 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1398/2783] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:08:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:08:44.811672 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e9105-3c5c-4a7b-b0c7-ea2f70d1327f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:44.813290 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7d0e9105-3c5c-4a7b-b0c7-ea2f70d1327f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:44.872992 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e9105-3c5c-4a7b-b0c7-ea2f70d1327f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:44.876794 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72429f42-a791-432b-8adb-3c7b238c1317 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:44.877419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72429f42-a791-432b-8adb-3c7b238c1317 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:44.877706 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72429f42-a791-432b-8adb-3c7b238c1317 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:44.877982 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72429f42-a791-432b-8adb-3c7b238c1317 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:44.879804 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-72429f42-a791-432b-8adb-3c7b238c1317 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:44.944023 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72429f42-a791-432b-8adb-3c7b238c1317 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:44.945448 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72429f42-a791-432b-8adb-3c7b238c1317 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:44.954991 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1386/2784] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:08:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:08:48.881358 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2530428-aa7c-4ffc-a019-107df6b5e873 service nova] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:48.882886 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e2530428-aa7c-4ffc-a019-107df6b5e873 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:48.898772 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2530428-aa7c-4ffc-a019-107df6b5e873 service nova] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:48.903538 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b5f51-67ff-41e3-bd4f-06935943457b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:48.904197 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b5f51-67ff-41e3-bd4f-06935943457b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:48.904481 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b5f51-67ff-41e3-bd4f-06935943457b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:48.904757 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b5f51-67ff-41e3-bd4f-06935943457b admin admin] PATH_INFO: `/v3/users/df675a52f9b245198897842f7a6ecd13` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:48.905957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b5f51-67ff-41e3-bd4f-06935943457b admin admin] RBAC: Authorizing `identity:delete_user(user_id=df675a52f9b245198897842f7a6ecd13)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:48.908079 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b5f51-67ff-41e3-bd4f-06935943457b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:49.217448 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0b0b5f51-67ff-41e3-bd4f-06935943457b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df675a52f9b245198897842f7a6ecd13'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:49.218126 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0b0b5f51-67ff-41e3-bd4f-06935943457b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df675a52f9b245198897842f7a6ecd13'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:49.231140 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0b0b5f51-67ff-41e3-bd4f-06935943457b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df675a52f9b245198897842f7a6ecd13'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:49.238333 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0b0b5f51-67ff-41e3-bd4f-06935943457b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df675a52f9b245198897842f7a6ecd13'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:49.239123 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0b0b5f51-67ff-41e3-bd4f-06935943457b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df675a52f9b245198897842f7a6ecd13'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:49.243203 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:08:49.379366 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1399/2785] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:08:48 2022] DELETE /identity/v3/users/df675a52f9b245198897842f7a6ecd13 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:08:49.676503 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72429f42-a791-432b-8adb-3c7b238c1317 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:49.687311 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72429f42-a791-432b-8adb-3c7b238c1317 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:49.745835 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c74caa-ddda-4153-a7e2-b9b6ac3b1c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:49.746483 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c74caa-ddda-4153-a7e2-b9b6ac3b1c22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:49.746766 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c74caa-ddda-4153-a7e2-b9b6ac3b1c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:49.747041 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c74caa-ddda-4153-a7e2-b9b6ac3b1c22 admin admin] PATH_INFO: `/v3/projects/7f82b8853b874e61820108bc16e8dc41` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:49.748230 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c74caa-ddda-4153-a7e2-b9b6ac3b1c22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f82b8853b874e61820108bc16e8dc41)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:49.750176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c74caa-ddda-4153-a7e2-b9b6ac3b1c22 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:49.909001 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-23c74caa-ddda-4153-a7e2-b9b6ac3b1c22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f82b8853b874e61820108bc16e8dc41', 'request_id': 'req-23c74caa-ddda-4153-a7e2-b9b6ac3b1c22'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:49.910982 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:08:49.915343 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-23c74caa-ddda-4153-a7e2-b9b6ac3b1c22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f82b8853b874e61820108bc16e8dc41', 'request_id': 'req-23c74caa-ddda-4153-a7e2-b9b6ac3b1c22'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:50.045194 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-23c74caa-ddda-4153-a7e2-b9b6ac3b1c22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f82b8853b874e61820108bc16e8dc41', 'request_id': 'req-23c74caa-ddda-4153-a7e2-b9b6ac3b1c22'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:50.058323 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-23c74caa-ddda-4153-a7e2-b9b6ac3b1c22 admin admin] The token cache is being invalidate because project 7f82b8853b874e61820108bc16e8dc41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:08:50.058617 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-23c74caa-ddda-4153-a7e2-b9b6ac3b1c22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:50.059262 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-23c74caa-ddda-4153-a7e2-b9b6ac3b1c22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:50.060861 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1387/2786] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:08:49 2022] DELETE /identity/v3/projects/7f82b8853b874e61820108bc16e8dc41 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:08:50.065931 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617e1d1-e2ec-4449-b239-a072aa9ea0a0 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:50.066380 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617e1d1-e2ec-4449-b239-a072aa9ea0a0 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:50.066506 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617e1d1-e2ec-4449-b239-a072aa9ea0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:50.086149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-b617e1d1-e2ec-4449-b239-a072aa9ea0a0 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:08:50.087348 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b617e1d1-e2ec-4449-b239-a072aa9ea0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:50.149418 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1400/2787] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:08:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:08:50.157641 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1942d-2aad-4ad5-aee1-e05c4ee888d9 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:50.157928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1942d-2aad-4ad5-aee1-e05c4ee888d9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:50.158209 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1942d-2aad-4ad5-aee1-e05c4ee888d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:50.179278 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a9a1942d-2aad-4ad5-aee1-e05c4ee888d9 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:08:50.180569 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a9a1942d-2aad-4ad5-aee1-e05c4ee888d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:50.188537 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1388/2788] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:08:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:08:50.192817 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617e1d1-e2ec-4449-b239-a072aa9ea0a0 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:50.202486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617e1d1-e2ec-4449-b239-a072aa9ea0a0 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:50.207082 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99460e-9e8c-430a-b450-84d5f1aaf16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:50.207714 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99460e-9e8c-430a-b450-84d5f1aaf16c admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:50.207999 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99460e-9e8c-430a-b450-84d5f1aaf16c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:50.208302 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99460e-9e8c-430a-b450-84d5f1aaf16c admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:50.208991 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99460e-9e8c-430a-b450-84d5f1aaf16c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:08:50.209324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99460e-9e8c-430a-b450-84d5f1aaf16c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:50.210919 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99460e-9e8c-430a-b450-84d5f1aaf16c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:50.222515 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1401/2789] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 15:08:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:08:50.227294 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1942d-2aad-4ad5-aee1-e05c4ee888d9 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:50.236560 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1942d-2aad-4ad5-aee1-e05c4ee888d9 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:50.241861 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2479fa2-c245-46f0-8922-4d575cc5685d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:50.243277 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2479fa2-c245-46f0-8922-4d575cc5685d admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:50.243277 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2479fa2-c245-46f0-8922-4d575cc5685d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:50.243277 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2479fa2-c245-46f0-8922-4d575cc5685d admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:50.244469 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2479fa2-c245-46f0-8922-4d575cc5685d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:50.245666 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2479fa2-c245-46f0-8922-4d575cc5685d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:50.369251 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1389/2790] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 15:08:50 2022] POST /identity/v3/projects => generated 383 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:08:50.373376 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99460e-9e8c-430a-b450-84d5f1aaf16c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:50.383953 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99460e-9e8c-430a-b450-84d5f1aaf16c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:50.441472 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245770ee-5012-478d-8894-0990404410a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:50.442099 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245770ee-5012-478d-8894-0990404410a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:50.442379 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245770ee-5012-478d-8894-0990404410a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:50.442654 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245770ee-5012-478d-8894-0990404410a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:50.443327 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245770ee-5012-478d-8894-0990404410a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:50.445321 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245770ee-5012-478d-8894-0990404410a9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:50.555118 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1402/2791] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 15:08:50 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:08:50.559574 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2479fa2-c245-46f0-8922-4d575cc5685d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:50.570013 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2479fa2-c245-46f0-8922-4d575cc5685d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:50.574606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e3cb4-9600-403b-8260-6929a717891f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:50.575486 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e3cb4-9600-403b-8260-6929a717891f admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:50.579619 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e3cb4-9600-403b-8260-6929a717891f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:50.579619 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e3cb4-9600-403b-8260-6929a717891f admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:50.579619 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e3cb4-9600-403b-8260-6929a717891f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:50.579619 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e3cb4-9600-403b-8260-6929a717891f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:50.589125 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1390/2792] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:08:50.593077 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245770ee-5012-478d-8894-0990404410a9 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:50.603924 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245770ee-5012-478d-8894-0990404410a9 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:50.608760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc48aa-f60d-4ccb-88e8-13d4c506d626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:50.614065 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc48aa-f60d-4ccb-88e8-13d4c506d626 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:50.614065 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc48aa-f60d-4ccb-88e8-13d4c506d626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:50.614065 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc48aa-f60d-4ccb-88e8-13d4c506d626 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:50.614065 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc48aa-f60d-4ccb-88e8-13d4c506d626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:50.614065 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc48aa-f60d-4ccb-88e8-13d4c506d626 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:50.623322 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1403/2793] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:08:50.627733 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e3cb4-9600-403b-8260-6929a717891f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:50.638367 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e3cb4-9600-403b-8260-6929a717891f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:50.642881 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a293b-5fc9-4e75-85f0-e7cd6d4ffbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:50.643615 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a293b-5fc9-4e75-85f0-e7cd6d4ffbce admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:50.643898 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a293b-5fc9-4e75-85f0-e7cd6d4ffbce admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:50.644437 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a293b-5fc9-4e75-85f0-e7cd6d4ffbce admin admin] PATH_INFO: `/v3/projects/e06910a243dd43d7976f10a6b87dba00/users/efbd1801fa1f42cc8e279152c2e036d2/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:50.659187 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a293b-5fc9-4e75-85f0-e7cd6d4ffbce admin admin] RBAC: Authorizing `identity:create_grant(project_id=e06910a243dd43d7976f10a6b87dba00, user_id=efbd1801fa1f42cc8e279152c2e036d2, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:50.661986 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a293b-5fc9-4e75-85f0-e7cd6d4ffbce admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:50.779612 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1391/2794] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:08:50 2022] PUT /identity/v3/projects/e06910a243dd43d7976f10a6b87dba00/users/efbd1801fa1f42cc8e279152c2e036d2/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:08:50.790141 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69a3d6-d7e7-439d-be2c-e4bfc7016947 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:50.790447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69a3d6-d7e7-439d-be2c-e4bfc7016947 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:50.795786 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69a3d6-d7e7-439d-be2c-e4bfc7016947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:50.818524 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0c69a3d6-d7e7-439d-be2c-e4bfc7016947 None None] MFA Rules not processed for user `efbd1801fa1f42cc8e279152c2e036d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:08:50.819785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0c69a3d6-d7e7-439d-be2c-e4bfc7016947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:50.872104 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:08:50.892353 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1404/2795] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:08:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:08:50.902014 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a293b-5fc9-4e75-85f0-e7cd6d4ffbce admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:50.904172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-701a293b-5fc9-4e75-85f0-e7cd6d4ffbce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:50.980597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a293b-5fc9-4e75-85f0-e7cd6d4ffbce admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:50.986348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9113e-d397-4e42-b485-306b51ca1255 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:50.986967 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9113e-d397-4e42-b485-306b51ca1255 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:50.987253 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9113e-d397-4e42-b485-306b51ca1255 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:50.987573 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9113e-d397-4e42-b485-306b51ca1255 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:50.996307 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f9113e-d397-4e42-b485-306b51ca1255 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:50.997936 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f9113e-d397-4e42-b485-306b51ca1255 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:51.007386 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1392/2796] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:08:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:09:00.959099 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:01.003068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=67,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:01.549198 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69a3d6-d7e7-439d-be2c-e4bfc7016947 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:01.613620 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69a3d6-d7e7-439d-be2c-e4bfc7016947 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:01.618725 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a5a47-bd45-464f-9578-5b4943f17fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:01.619385 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a5a47-bd45-464f-9578-5b4943f17fe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:01.619691 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a5a47-bd45-464f-9578-5b4943f17fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:01.619971 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a5a47-bd45-464f-9578-5b4943f17fe2 admin admin] PATH_INFO: `/v3/users/efbd1801fa1f42cc8e279152c2e036d2` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:01.621162 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a5a47-bd45-464f-9578-5b4943f17fe2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efbd1801fa1f42cc8e279152c2e036d2)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:01.623171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a5a47-bd45-464f-9578-5b4943f17fe2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:01.938299 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3e2a5a47-bd45-464f-9578-5b4943f17fe2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efbd1801fa1f42cc8e279152c2e036d2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:01.938957 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3e2a5a47-bd45-464f-9578-5b4943f17fe2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efbd1801fa1f42cc8e279152c2e036d2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:01.953946 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3e2a5a47-bd45-464f-9578-5b4943f17fe2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efbd1801fa1f42cc8e279152c2e036d2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:01.963284 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3e2a5a47-bd45-464f-9578-5b4943f17fe2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efbd1801fa1f42cc8e279152c2e036d2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:01.964063 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3e2a5a47-bd45-464f-9578-5b4943f17fe2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efbd1801fa1f42cc8e279152c2e036d2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:02.105543 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1405/2797] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:09:01 2022] DELETE /identity/v3/users/efbd1801fa1f42cc8e279152c2e036d2 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:02.110136 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d2236-02b7-4f3c-a183-9552b6a8e8f5 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:02.110767 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d2236-02b7-4f3c-a183-9552b6a8e8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:02.110908 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d2236-02b7-4f3c-a183-9552b6a8e8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:02.131571 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9f1d2236-02b7-4f3c-a183-9552b6a8e8f5 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:02.132764 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9f1d2236-02b7-4f3c-a183-9552b6a8e8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:02.193120 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1393/2798] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:09:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:02.203253 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a5a47-bd45-464f-9578-5b4943f17fe2 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:02.204773 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3e2a5a47-bd45-464f-9578-5b4943f17fe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:02.264494 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a5a47-bd45-464f-9578-5b4943f17fe2 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:02.268403 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3157384e-9542-4fb7-b633-29b0ecff58dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:02.269115 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157384e-9542-4fb7-b633-29b0ecff58dd service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:02.269714 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157384e-9542-4fb7-b633-29b0ecff58dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:02.269992 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157384e-9542-4fb7-b633-29b0ecff58dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:02.280036 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3157384e-9542-4fb7-b633-29b0ecff58dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:02.281517 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3157384e-9542-4fb7-b633-29b0ecff58dd service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:02.291007 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1406/2799] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:09:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:09:02.871056 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d2236-02b7-4f3c-a183-9552b6a8e8f5 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:02.872521 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9f1d2236-02b7-4f3c-a183-9552b6a8e8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:02.884696 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d2236-02b7-4f3c-a183-9552b6a8e8f5 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:02.889611 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea72f60-f865-49f7-8e3b-0314afffd685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:02.890248 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea72f60-f865-49f7-8e3b-0314afffd685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:02.890520 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea72f60-f865-49f7-8e3b-0314afffd685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:02.890794 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea72f60-f865-49f7-8e3b-0314afffd685 admin admin] PATH_INFO: `/v3/projects/e06910a243dd43d7976f10a6b87dba00` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:02.891991 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea72f60-f865-49f7-8e3b-0314afffd685 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e06910a243dd43d7976f10a6b87dba00)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:02.894053 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea72f60-f865-49f7-8e3b-0314afffd685 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:03.034740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-eea72f60-f865-49f7-8e3b-0314afffd685 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e06910a243dd43d7976f10a6b87dba00', 'request_id': 'req-eea72f60-f865-49f7-8e3b-0314afffd685'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:03.040610 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-eea72f60-f865-49f7-8e3b-0314afffd685 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e06910a243dd43d7976f10a6b87dba00', 'request_id': 'req-eea72f60-f865-49f7-8e3b-0314afffd685'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:03.179506 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-eea72f60-f865-49f7-8e3b-0314afffd685 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e06910a243dd43d7976f10a6b87dba00', 'request_id': 'req-eea72f60-f865-49f7-8e3b-0314afffd685'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:03.191842 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-eea72f60-f865-49f7-8e3b-0314afffd685 admin admin] The token cache is being invalidate because project e06910a243dd43d7976f10a6b87dba00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:09:03.192131 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-eea72f60-f865-49f7-8e3b-0314afffd685 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:03.193461 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-eea72f60-f865-49f7-8e3b-0314afffd685 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:03.195117 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1394/2800] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:09:02 2022] DELETE /identity/v3/projects/e06910a243dd43d7976f10a6b87dba00 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:03.200626 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b6b28-0109-47d3-87f2-a02919cdb62b None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:03.201226 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b6b28-0109-47d3-87f2-a02919cdb62b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:03.201226 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b6b28-0109-47d3-87f2-a02919cdb62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:03.220777 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c88b6b28-0109-47d3-87f2-a02919cdb62b None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:03.221980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c88b6b28-0109-47d3-87f2-a02919cdb62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:03.240249 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:03.284083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1407/2801] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:09:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:09:03.350764 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66fbf8-55d0-4e3a-9311-766e5a0b7345 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:03.351048 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66fbf8-55d0-4e3a-9311-766e5a0b7345 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:03.351550 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66fbf8-55d0-4e3a-9311-766e5a0b7345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:03.372892 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-7b66fbf8-55d0-4e3a-9311-766e5a0b7345 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:03.374119 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7b66fbf8-55d0-4e3a-9311-766e5a0b7345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:03.383002 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1395/2802] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:09:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:03.387341 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b6b28-0109-47d3-87f2-a02919cdb62b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:03.397458 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b6b28-0109-47d3-87f2-a02919cdb62b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:03.402215 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb1f0e5-f17d-44d2-bd6e-e9c5b0529ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:03.402852 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1f0e5-f17d-44d2-bd6e-e9c5b0529ddf admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:03.403129 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1f0e5-f17d-44d2-bd6e-e9c5b0529ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:03.403490 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1f0e5-f17d-44d2-bd6e-e9c5b0529ddf admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:03.404189 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb1f0e5-f17d-44d2-bd6e-e9c5b0529ddf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:09:03.404508 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb1f0e5-f17d-44d2-bd6e-e9c5b0529ddf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:03.406302 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb1f0e5-f17d-44d2-bd6e-e9c5b0529ddf admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:03.419076 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1408/2803] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 15:09:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:09:03.424389 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66fbf8-55d0-4e3a-9311-766e5a0b7345 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:03.437238 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66fbf8-55d0-4e3a-9311-766e5a0b7345 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:03.442912 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbf1de-5fca-4d8e-b1d1-f2e7eaa6ced7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:03.443717 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbf1de-5fca-4d8e-b1d1-f2e7eaa6ced7 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:03.444032 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbf1de-5fca-4d8e-b1d1-f2e7eaa6ced7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:03.444372 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbf1de-5fca-4d8e-b1d1-f2e7eaa6ced7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:03.445365 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabbf1de-5fca-4d8e-b1d1-f2e7eaa6ced7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:03.447629 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabbf1de-5fca-4d8e-b1d1-f2e7eaa6ced7 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:03.590763 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1396/2804] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 15:09:03 2022] POST /identity/v3/projects => generated 401 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:09:03.595660 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb1f0e5-f17d-44d2-bd6e-e9c5b0529ddf admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:03.611751 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb1f0e5-f17d-44d2-bd6e-e9c5b0529ddf admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:03.675087 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068133d6-c7d2-437e-814f-a417f85bcec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:03.675741 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068133d6-c7d2-437e-814f-a417f85bcec0 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:03.676025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068133d6-c7d2-437e-814f-a417f85bcec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:03.676300 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068133d6-c7d2-437e-814f-a417f85bcec0 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:03.676987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068133d6-c7d2-437e-814f-a417f85bcec0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:03.679517 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068133d6-c7d2-437e-814f-a417f85bcec0 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:03.767891 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1409/2805] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 15:09:03 2022] POST /identity/v3/users => generated 301 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:09:03.773496 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbf1de-5fca-4d8e-b1d1-f2e7eaa6ced7 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:03.784300 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbf1de-5fca-4d8e-b1d1-f2e7eaa6ced7 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:03.789711 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b528ebe-756a-4d98-b59f-3ce63ba10180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:03.790315 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b528ebe-756a-4d98-b59f-3ce63ba10180 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:03.790599 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b528ebe-756a-4d98-b59f-3ce63ba10180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:03.790874 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b528ebe-756a-4d98-b59f-3ce63ba10180 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:03.791592 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b528ebe-756a-4d98-b59f-3ce63ba10180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:03.793275 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b528ebe-756a-4d98-b59f-3ce63ba10180 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:03.804018 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1397/2806] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:09:03.808252 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068133d6-c7d2-437e-814f-a417f85bcec0 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:03.818895 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068133d6-c7d2-437e-814f-a417f85bcec0 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:03.823818 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e812131-6cfb-4193-816f-172cc9dd4fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:03.824472 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e812131-6cfb-4193-816f-172cc9dd4fb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:03.824785 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e812131-6cfb-4193-816f-172cc9dd4fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:03.825092 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e812131-6cfb-4193-816f-172cc9dd4fb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:03.825815 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e812131-6cfb-4193-816f-172cc9dd4fb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:03.827446 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e812131-6cfb-4193-816f-172cc9dd4fb5 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:03.838847 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1410/2807] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:09:03.843065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b528ebe-756a-4d98-b59f-3ce63ba10180 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:03.853857 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b528ebe-756a-4d98-b59f-3ce63ba10180 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:03.858496 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d510e3-23bf-4816-b1c9-82a29bbc1ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:03.859151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d510e3-23bf-4816-b1c9-82a29bbc1ad0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:03.859458 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d510e3-23bf-4816-b1c9-82a29bbc1ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:03.859740 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d510e3-23bf-4816-b1c9-82a29bbc1ad0 admin admin] PATH_INFO: `/v3/projects/08aceefaa9c449dba11e55e21af66d34/users/a95503041392484a94cd025bf7dcd8de/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:03.874290 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d510e3-23bf-4816-b1c9-82a29bbc1ad0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08aceefaa9c449dba11e55e21af66d34, user_id=a95503041392484a94cd025bf7dcd8de, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:03.876969 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d510e3-23bf-4816-b1c9-82a29bbc1ad0 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:04.003420 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1398/2808] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:09:03 2022] PUT /identity/v3/projects/08aceefaa9c449dba11e55e21af66d34/users/a95503041392484a94cd025bf7dcd8de/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:04.008629 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86caeb-0f40-46a5-ad87-22e8b4d63699 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:04.008929 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86caeb-0f40-46a5-ad87-22e8b4d63699 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:04.009201 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86caeb-0f40-46a5-ad87-22e8b4d63699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:04.030596 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6f86caeb-0f40-46a5-ad87-22e8b4d63699 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:04.031725 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6f86caeb-0f40-46a5-ad87-22e8b4d63699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:04.091493 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1411/2809] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:09:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:04.102306 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d510e3-23bf-4816-b1c9-82a29bbc1ad0 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:04.104164 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-49d510e3-23bf-4816-b1c9-82a29bbc1ad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:04.128467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:04.165582 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d510e3-23bf-4816-b1c9-82a29bbc1ad0 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:04.169845 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c1db94-7799-443d-80d0-68736a646b64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:04.170788 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c1db94-7799-443d-80d0-68736a646b64 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:04.171106 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c1db94-7799-443d-80d0-68736a646b64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:04.171506 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c1db94-7799-443d-80d0-68736a646b64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:04.184171 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c1db94-7799-443d-80d0-68736a646b64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:04.185876 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c1db94-7799-443d-80d0-68736a646b64 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:04.198575 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1399/2810] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:09:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:09:05.914145 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd1059-f1fb-4235-bf14-93f4905ebeef None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:05.914771 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd1059-f1fb-4235-bf14-93f4905ebeef None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:05.915032 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd1059-f1fb-4235-bf14-93f4905ebeef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:05.951267 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3dfd1059-f1fb-4235-bf14-93f4905ebeef None None] MFA Rules not processed for user `a95503041392484a94cd025bf7dcd8de`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:05.952940 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3dfd1059-f1fb-4235-bf14-93f4905ebeef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:05.972848 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:06.028808 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1412/2811] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:09:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:06.033319 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c1db94-7799-443d-80d0-68736a646b64 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:06.044180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c1db94-7799-443d-80d0-68736a646b64 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:06.049214 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf16b8-56c4-4ebd-9abb-cff8f9a42703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:06.049837 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf16b8-56c4-4ebd-9abb-cff8f9a42703 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:06.050128 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf16b8-56c4-4ebd-9abb-cff8f9a42703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:06.050429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf16b8-56c4-4ebd-9abb-cff8f9a42703 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:06.051068 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf16b8-56c4-4ebd-9abb-cff8f9a42703 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:06.053126 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf16b8-56c4-4ebd-9abb-cff8f9a42703 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:06.195810 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1400/2812] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 15:09:06 2022] POST /identity/v3/projects => generated 405 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:09:06.200145 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd1059-f1fb-4235-bf14-93f4905ebeef None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:06.212009 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd1059-f1fb-4235-bf14-93f4905ebeef None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:06.269257 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af928f-d9eb-441e-9ac2-5dc69e5a1546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:06.270464 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af928f-d9eb-441e-9ac2-5dc69e5a1546 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:06.271324 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af928f-d9eb-441e-9ac2-5dc69e5a1546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:06.271635 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af928f-d9eb-441e-9ac2-5dc69e5a1546 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:06.272358 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9af928f-d9eb-441e-9ac2-5dc69e5a1546 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:06.274286 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9af928f-d9eb-441e-9ac2-5dc69e5a1546 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:06.400786 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1413/2813] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 15:09:06 2022] POST /identity/v3/users => generated 309 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:09:06.404757 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf16b8-56c4-4ebd-9abb-cff8f9a42703 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:06.415076 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf16b8-56c4-4ebd-9abb-cff8f9a42703 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:06.419785 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248139c-9977-4220-b436-efa5e95df6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:06.420411 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248139c-9977-4220-b436-efa5e95df6bb admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:06.420717 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248139c-9977-4220-b436-efa5e95df6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:06.420978 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248139c-9977-4220-b436-efa5e95df6bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:06.421670 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2248139c-9977-4220-b436-efa5e95df6bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:06.423275 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2248139c-9977-4220-b436-efa5e95df6bb admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:06.434360 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1401/2814] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:09:06.440659 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af928f-d9eb-441e-9ac2-5dc69e5a1546 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:06.452451 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af928f-d9eb-441e-9ac2-5dc69e5a1546 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:06.457672 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd713c64-a73b-4291-b544-f82fe18c7b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:06.459099 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd713c64-a73b-4291-b544-f82fe18c7b73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:06.459504 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd713c64-a73b-4291-b544-f82fe18c7b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:06.459864 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd713c64-a73b-4291-b544-f82fe18c7b73 admin admin] PATH_INFO: `/v3/projects/c4bf420c52d44525a94b69625a506a8e/users/f89c202b42724d42bb1c45fd16960c27/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:06.478144 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd713c64-a73b-4291-b544-f82fe18c7b73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4bf420c52d44525a94b69625a506a8e, user_id=f89c202b42724d42bb1c45fd16960c27, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:06.480890 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd713c64-a73b-4291-b544-f82fe18c7b73 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:06.625842 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1414/2815] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:09:06 2022] PUT /identity/v3/projects/c4bf420c52d44525a94b69625a506a8e/users/f89c202b42724d42bb1c45fd16960c27/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:06.629791 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248139c-9977-4220-b436-efa5e95df6bb admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:06.640331 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248139c-9977-4220-b436-efa5e95df6bb admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:06.699840 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43701787-7581-4d71-bb3d-262759a431f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:06.700525 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43701787-7581-4d71-bb3d-262759a431f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:06.700820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43701787-7581-4d71-bb3d-262759a431f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:06.701120 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43701787-7581-4d71-bb3d-262759a431f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:06.701847 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43701787-7581-4d71-bb3d-262759a431f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:06.703678 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43701787-7581-4d71-bb3d-262759a431f7 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:06.714824 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1402/2816] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:09:06.719674 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd713c64-a73b-4291-b544-f82fe18c7b73 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:06.730827 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd713c64-a73b-4291-b544-f82fe18c7b73 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:06.735620 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d1b7f-543d-4c93-af75-17b898b46c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:06.736272 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d1b7f-543d-4c93-af75-17b898b46c71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:06.736554 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d1b7f-543d-4c93-af75-17b898b46c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:06.736833 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d1b7f-543d-4c93-af75-17b898b46c71 admin admin] PATH_INFO: `/v3/projects/c4bf420c52d44525a94b69625a506a8e/users/f89c202b42724d42bb1c45fd16960c27/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:06.738756 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d1b7f-543d-4c93-af75-17b898b46c71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4bf420c52d44525a94b69625a506a8e, user_id=f89c202b42724d42bb1c45fd16960c27, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:06.741989 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d1b7f-543d-4c93-af75-17b898b46c71 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:06.753048 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1415/2817] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:09:06 2022] PUT /identity/v3/projects/c4bf420c52d44525a94b69625a506a8e/users/f89c202b42724d42bb1c45fd16960c27/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:08.265482 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59084c92-34fa-4410-bc37-b1f1bc713d83 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:08.266073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59084c92-34fa-4410-bc37-b1f1bc713d83 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:08.266073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59084c92-34fa-4410-bc37-b1f1bc713d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:08.296020 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-59084c92-34fa-4410-bc37-b1f1bc713d83 None None] MFA Rules not processed for user `f89c202b42724d42bb1c45fd16960c27`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:08.299529 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-59084c92-34fa-4410-bc37-b1f1bc713d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:08.370736 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1403/2818] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:09:08 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:09.843884 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d1b7f-543d-4c93-af75-17b898b46c71 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:09.915515 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d1b7f-543d-4c93-af75-17b898b46c71 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:09.920176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420e87b-3d6c-4e49-8442-d8aa63586754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:09.920874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420e87b-3d6c-4e49-8442-d8aa63586754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:09.921106 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420e87b-3d6c-4e49-8442-d8aa63586754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:09.921386 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420e87b-3d6c-4e49-8442-d8aa63586754 admin admin] PATH_INFO: `/v3/users/a95503041392484a94cd025bf7dcd8de` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:09.922580 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f420e87b-3d6c-4e49-8442-d8aa63586754 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a95503041392484a94cd025bf7dcd8de)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:09.924616 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f420e87b-3d6c-4e49-8442-d8aa63586754 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:09.937708 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:10.153682 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f420e87b-3d6c-4e49-8442-d8aa63586754 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a95503041392484a94cd025bf7dcd8de'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:10.154747 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f420e87b-3d6c-4e49-8442-d8aa63586754 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a95503041392484a94cd025bf7dcd8de'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:10.169083 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f420e87b-3d6c-4e49-8442-d8aa63586754 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a95503041392484a94cd025bf7dcd8de'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:10.176172 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f420e87b-3d6c-4e49-8442-d8aa63586754 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a95503041392484a94cd025bf7dcd8de'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:10.176832 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f420e87b-3d6c-4e49-8442-d8aa63586754 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a95503041392484a94cd025bf7dcd8de'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:10.275072 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1416/2819] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:09:09 2022] DELETE /identity/v3/users/a95503041392484a94cd025bf7dcd8de => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:10.547316 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59084c92-34fa-4410-bc37-b1f1bc713d83 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:10.559010 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59084c92-34fa-4410-bc37-b1f1bc713d83 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:10.567444 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:10.619303 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396886bf-1fa8-43ff-a16c-abb118d08634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:10.620108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396886bf-1fa8-43ff-a16c-abb118d08634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:10.620434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396886bf-1fa8-43ff-a16c-abb118d08634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:10.620759 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396886bf-1fa8-43ff-a16c-abb118d08634 admin admin] PATH_INFO: `/v3/projects/08aceefaa9c449dba11e55e21af66d34` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:10.622004 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396886bf-1fa8-43ff-a16c-abb118d08634 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08aceefaa9c449dba11e55e21af66d34)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:10.624042 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396886bf-1fa8-43ff-a16c-abb118d08634 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:10.780738 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-396886bf-1fa8-43ff-a16c-abb118d08634 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08aceefaa9c449dba11e55e21af66d34', 'request_id': 'req-396886bf-1fa8-43ff-a16c-abb118d08634'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:10.786284 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-396886bf-1fa8-43ff-a16c-abb118d08634 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08aceefaa9c449dba11e55e21af66d34', 'request_id': 'req-396886bf-1fa8-43ff-a16c-abb118d08634'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:10.888010 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-396886bf-1fa8-43ff-a16c-abb118d08634 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08aceefaa9c449dba11e55e21af66d34', 'request_id': 'req-396886bf-1fa8-43ff-a16c-abb118d08634'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:10.901348 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-396886bf-1fa8-43ff-a16c-abb118d08634 admin admin] The token cache is being invalidate because project 08aceefaa9c449dba11e55e21af66d34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:09:10.908729 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-396886bf-1fa8-43ff-a16c-abb118d08634 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:10.908729 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-396886bf-1fa8-43ff-a16c-abb118d08634 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:10.908729 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1404/2820] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:09:10 2022] DELETE /identity/v3/projects/08aceefaa9c449dba11e55e21af66d34 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:10.909056 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420e87b-3d6c-4e49-8442-d8aa63586754 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:10.909503 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f420e87b-3d6c-4e49-8442-d8aa63586754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:10.923069 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420e87b-3d6c-4e49-8442-d8aa63586754 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:10.982788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4882dda-0fdd-4ac2-a9a3-74c1795f63f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:10.983461 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4882dda-0fdd-4ac2-a9a3-74c1795f63f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:10.983752 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4882dda-0fdd-4ac2-a9a3-74c1795f63f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:10.984059 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4882dda-0fdd-4ac2-a9a3-74c1795f63f8 admin admin] PATH_INFO: `/v3/users/f89c202b42724d42bb1c45fd16960c27` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:10.985251 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4882dda-0fdd-4ac2-a9a3-74c1795f63f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f89c202b42724d42bb1c45fd16960c27)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:10.987274 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4882dda-0fdd-4ac2-a9a3-74c1795f63f8 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:11.206618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f4882dda-0fdd-4ac2-a9a3-74c1795f63f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f89c202b42724d42bb1c45fd16960c27'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:11.383365 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f4882dda-0fdd-4ac2-a9a3-74c1795f63f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f89c202b42724d42bb1c45fd16960c27'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:11.383365 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f4882dda-0fdd-4ac2-a9a3-74c1795f63f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f89c202b42724d42bb1c45fd16960c27'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:11.383365 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f4882dda-0fdd-4ac2-a9a3-74c1795f63f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f89c202b42724d42bb1c45fd16960c27'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:11.383365 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f4882dda-0fdd-4ac2-a9a3-74c1795f63f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f89c202b42724d42bb1c45fd16960c27'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:11.446098 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1417/2821] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:09:10 2022] DELETE /identity/v3/users/f89c202b42724d42bb1c45fd16960c27 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:11.788077 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396886bf-1fa8-43ff-a16c-abb118d08634 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:11.790002 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-396886bf-1fa8-43ff-a16c-abb118d08634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:11.806913 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396886bf-1fa8-43ff-a16c-abb118d08634 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:11.871921 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef7b2f-46e6-41bf-b7ff-4902e0b9215c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:11.872573 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef7b2f-46e6-41bf-b7ff-4902e0b9215c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:11.872857 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef7b2f-46e6-41bf-b7ff-4902e0b9215c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:11.873135 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef7b2f-46e6-41bf-b7ff-4902e0b9215c admin admin] PATH_INFO: `/v3/projects/c4bf420c52d44525a94b69625a506a8e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:11.874351 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef7b2f-46e6-41bf-b7ff-4902e0b9215c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4bf420c52d44525a94b69625a506a8e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:11.876306 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef7b2f-46e6-41bf-b7ff-4902e0b9215c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:11.987811 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:11.991965 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ecef7b2f-46e6-41bf-b7ff-4902e0b9215c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4bf420c52d44525a94b69625a506a8e', 'request_id': 'req-ecef7b2f-46e6-41bf-b7ff-4902e0b9215c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:11.997463 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ecef7b2f-46e6-41bf-b7ff-4902e0b9215c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4bf420c52d44525a94b69625a506a8e', 'request_id': 'req-ecef7b2f-46e6-41bf-b7ff-4902e0b9215c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:12.099675 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ecef7b2f-46e6-41bf-b7ff-4902e0b9215c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4bf420c52d44525a94b69625a506a8e', 'request_id': 'req-ecef7b2f-46e6-41bf-b7ff-4902e0b9215c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:12.111687 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ecef7b2f-46e6-41bf-b7ff-4902e0b9215c admin admin] The token cache is being invalidate because project c4bf420c52d44525a94b69625a506a8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:09:12.113930 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ecef7b2f-46e6-41bf-b7ff-4902e0b9215c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:12.113930 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ecef7b2f-46e6-41bf-b7ff-4902e0b9215c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:12.114246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1405/2822] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:09:11 2022] DELETE /identity/v3/projects/c4bf420c52d44525a94b69625a506a8e => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:12.119509 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf1993-e6e9-4f4e-9f4b-1d744d104d5a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:12.120036 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf1993-e6e9-4f4e-9f4b-1d744d104d5a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:12.120177 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf1993-e6e9-4f4e-9f4b-1d744d104d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:12.141574 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-5acf1993-e6e9-4f4e-9f4b-1d744d104d5a None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:12.142900 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5acf1993-e6e9-4f4e-9f4b-1d744d104d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:12.148023 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:12.207282 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1418/2823] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:09:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:09:12.216904 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b93457-284a-415e-87d7-88848d1b36f3 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:12.217246 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b93457-284a-415e-87d7-88848d1b36f3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:12.217525 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b93457-284a-415e-87d7-88848d1b36f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:12.238289 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b4b93457-284a-415e-87d7-88848d1b36f3 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:12.240016 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b4b93457-284a-415e-87d7-88848d1b36f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:12.247990 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1406/2824] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:09:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:12.252219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf1993-e6e9-4f4e-9f4b-1d744d104d5a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:12.262726 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf1993-e6e9-4f4e-9f4b-1d744d104d5a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:12.268092 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ee84f-1d45-425e-bd37-9640c52153e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:12.268819 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ee84f-1d45-425e-bd37-9640c52153e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:12.269238 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ee84f-1d45-425e-bd37-9640c52153e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:12.269563 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ee84f-1d45-425e-bd37-9640c52153e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:12.270853 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ee84f-1d45-425e-bd37-9640c52153e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:09:12.271253 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ee84f-1d45-425e-bd37-9640c52153e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:12.273219 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ee84f-1d45-425e-bd37-9640c52153e3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:12.287562 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1419/2825] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 15:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:09:12.291927 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b93457-284a-415e-87d7-88848d1b36f3 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:12.301693 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b93457-284a-415e-87d7-88848d1b36f3 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:12.307538 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9273e-ef27-4c8c-a6bb-8a839cb0a791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:12.309227 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9273e-ef27-4c8c-a6bb-8a839cb0a791 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:12.309515 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9273e-ef27-4c8c-a6bb-8a839cb0a791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:12.309811 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9273e-ef27-4c8c-a6bb-8a839cb0a791 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:12.310584 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f9273e-ef27-4c8c-a6bb-8a839cb0a791 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:12.312844 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f9273e-ef27-4c8c-a6bb-8a839cb0a791 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:12.479114 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1407/2826] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 15:09:12 2022] POST /identity/v3/projects => generated 379 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:09:12.483468 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ee84f-1d45-425e-bd37-9640c52153e3 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:12.494648 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ee84f-1d45-425e-bd37-9640c52153e3 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:12.550873 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a821ee1-8d1b-462b-98a3-097622e0ee8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:12.551559 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a821ee1-8d1b-462b-98a3-097622e0ee8d admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:12.551855 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a821ee1-8d1b-462b-98a3-097622e0ee8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:12.552786 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a821ee1-8d1b-462b-98a3-097622e0ee8d admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:12.553431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a821ee1-8d1b-462b-98a3-097622e0ee8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:12.555333 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a821ee1-8d1b-462b-98a3-097622e0ee8d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:12.686105 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1420/2827] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 15:09:12 2022] POST /identity/v3/users => generated 290 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:09:12.693762 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9273e-ef27-4c8c-a6bb-8a839cb0a791 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:12.704670 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9273e-ef27-4c8c-a6bb-8a839cb0a791 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:12.709310 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7738af-13b2-4e99-aefc-88ee4aa043f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:12.709924 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7738af-13b2-4e99-aefc-88ee4aa043f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:12.710205 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7738af-13b2-4e99-aefc-88ee4aa043f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:12.710482 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7738af-13b2-4e99-aefc-88ee4aa043f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:12.711176 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7738af-13b2-4e99-aefc-88ee4aa043f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:12.712843 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7738af-13b2-4e99-aefc-88ee4aa043f1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:12.723420 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1408/2828] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:09:12.728388 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a821ee1-8d1b-462b-98a3-097622e0ee8d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:12.738743 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a821ee1-8d1b-462b-98a3-097622e0ee8d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:12.744323 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eff9b6-a19a-4e77-835b-845fed2ee119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:12.744967 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eff9b6-a19a-4e77-835b-845fed2ee119 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:12.745252 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eff9b6-a19a-4e77-835b-845fed2ee119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:12.745559 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eff9b6-a19a-4e77-835b-845fed2ee119 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:12.746337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eff9b6-a19a-4e77-835b-845fed2ee119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:12.748121 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eff9b6-a19a-4e77-835b-845fed2ee119 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:12.759574 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1421/2829] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:09:12.763982 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7738af-13b2-4e99-aefc-88ee4aa043f1 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:12.774946 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7738af-13b2-4e99-aefc-88ee4aa043f1 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:12.779630 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605f422-d662-4007-b2d0-f3c1a4c19f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:12.780364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605f422-d662-4007-b2d0-f3c1a4c19f8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:12.780645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605f422-d662-4007-b2d0-f3c1a4c19f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:12.780924 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605f422-d662-4007-b2d0-f3c1a4c19f8b admin admin] PATH_INFO: `/v3/projects/dbb6c9f63b1848f3a480e7771e79a157/users/63c08b471ad84c5f8ca0b8b88e96b8f6/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:12.796499 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f605f422-d662-4007-b2d0-f3c1a4c19f8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbb6c9f63b1848f3a480e7771e79a157, user_id=63c08b471ad84c5f8ca0b8b88e96b8f6, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:12.799261 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f605f422-d662-4007-b2d0-f3c1a4c19f8b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:12.882998 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1409/2830] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:09:12 2022] PUT /identity/v3/projects/dbb6c9f63b1848f3a480e7771e79a157/users/63c08b471ad84c5f8ca0b8b88e96b8f6/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:12.894395 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673d2e0-6a58-4f34-af9b-4cb642a66468 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:12.896928 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673d2e0-6a58-4f34-af9b-4cb642a66468 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:12.896928 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673d2e0-6a58-4f34-af9b-4cb642a66468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:12.924225 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-2673d2e0-6a58-4f34-af9b-4cb642a66468 None None] MFA Rules not processed for user `63c08b471ad84c5f8ca0b8b88e96b8f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:12.925443 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2673d2e0-6a58-4f34-af9b-4cb642a66468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:12.991003 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1422/2831] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:09:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:12.995798 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605f422-d662-4007-b2d0-f3c1a4c19f8b admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:13.061984 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605f422-d662-4007-b2d0-f3c1a4c19f8b admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:13.066723 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97743900-9124-4621-8bc1-41597cc94550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:13.067340 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97743900-9124-4621-8bc1-41597cc94550 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:13.067646 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97743900-9124-4621-8bc1-41597cc94550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:13.067920 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97743900-9124-4621-8bc1-41597cc94550 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:13.068589 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97743900-9124-4621-8bc1-41597cc94550 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:13.070498 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97743900-9124-4621-8bc1-41597cc94550 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:13.213415 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1410/2832] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 15:09:12 2022] POST /identity/v3/projects => generated 381 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:09:13.217906 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673d2e0-6a58-4f34-af9b-4cb642a66468 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:13.224739 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:13.229397 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673d2e0-6a58-4f34-af9b-4cb642a66468 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:13.285554 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794976a-f5f1-4f03-a973-02df03949e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:13.286176 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794976a-f5f1-4f03-a973-02df03949e10 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:13.286461 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794976a-f5f1-4f03-a973-02df03949e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:13.286739 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794976a-f5f1-4f03-a973-02df03949e10 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:13.287437 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794976a-f5f1-4f03-a973-02df03949e10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:13.289366 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794976a-f5f1-4f03-a973-02df03949e10 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:13.414435 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1423/2833] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 15:09:13 2022] POST /identity/v3/users => generated 297 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:09:13.418573 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97743900-9124-4621-8bc1-41597cc94550 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:13.428803 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97743900-9124-4621-8bc1-41597cc94550 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:13.434960 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af5df8b-eaf5-4686-9fe6-8be21f4a13bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:13.434960 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5df8b-eaf5-4686-9fe6-8be21f4a13bf admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:13.434960 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5df8b-eaf5-4686-9fe6-8be21f4a13bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:13.434960 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5df8b-eaf5-4686-9fe6-8be21f4a13bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:13.435427 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af5df8b-eaf5-4686-9fe6-8be21f4a13bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:13.436973 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af5df8b-eaf5-4686-9fe6-8be21f4a13bf admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:13.448041 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1411/2834] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:09:13.452407 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794976a-f5f1-4f03-a973-02df03949e10 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:13.463148 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794976a-f5f1-4f03-a973-02df03949e10 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:13.467814 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab19cf0-74ad-4fbf-bb47-9dd9272f114c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:13.468474 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab19cf0-74ad-4fbf-bb47-9dd9272f114c admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:13.468754 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab19cf0-74ad-4fbf-bb47-9dd9272f114c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:13.469025 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab19cf0-74ad-4fbf-bb47-9dd9272f114c admin admin] PATH_INFO: `/v3/projects/fcce1fcf19bb4820957c5258a5e28126/users/f65349b7fda74746a250f29360235ef5/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:13.483760 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab19cf0-74ad-4fbf-bb47-9dd9272f114c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcce1fcf19bb4820957c5258a5e28126, user_id=f65349b7fda74746a250f29360235ef5, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:13.486515 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab19cf0-74ad-4fbf-bb47-9dd9272f114c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:13.613937 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1424/2835] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:09:13 2022] PUT /identity/v3/projects/fcce1fcf19bb4820957c5258a5e28126/users/f65349b7fda74746a250f29360235ef5/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:13.617999 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af5df8b-eaf5-4686-9fe6-8be21f4a13bf admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:13.628214 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af5df8b-eaf5-4686-9fe6-8be21f4a13bf admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:13.668813 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:13.688087 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b5a38-2917-4ab9-abe1-17b54cf43f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:13.688710 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b5a38-2917-4ab9-abe1-17b54cf43f87 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:13.688994 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b5a38-2917-4ab9-abe1-17b54cf43f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:13.689271 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b5a38-2917-4ab9-abe1-17b54cf43f87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:13.689965 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5b5a38-2917-4ab9-abe1-17b54cf43f87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:13.691586 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5b5a38-2917-4ab9-abe1-17b54cf43f87 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:13.702150 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1412/2836] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:09:13.707215 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab19cf0-74ad-4fbf-bb47-9dd9272f114c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:13.716877 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab19cf0-74ad-4fbf-bb47-9dd9272f114c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:13.721534 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f059b1-193d-40f1-9b83-455b337527f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:13.722202 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f059b1-193d-40f1-9b83-455b337527f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:13.722488 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f059b1-193d-40f1-9b83-455b337527f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:13.722775 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f059b1-193d-40f1-9b83-455b337527f3 admin admin] PATH_INFO: `/v3/projects/fcce1fcf19bb4820957c5258a5e28126/users/f65349b7fda74746a250f29360235ef5/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:13.725007 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f059b1-193d-40f1-9b83-455b337527f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcce1fcf19bb4820957c5258a5e28126, user_id=f65349b7fda74746a250f29360235ef5, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:13.727674 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f059b1-193d-40f1-9b83-455b337527f3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:13.737919 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1425/2837] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:09:13 2022] PUT /identity/v3/projects/fcce1fcf19bb4820957c5258a5e28126/users/f65349b7fda74746a250f29360235ef5/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:13.748698 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317294e-a2d2-450c-b809-55889b51065a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:13.748986 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317294e-a2d2-450c-b809-55889b51065a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:13.749264 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317294e-a2d2-450c-b809-55889b51065a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:13.777671 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-4317294e-a2d2-450c-b809-55889b51065a None None] MFA Rules not processed for user `f65349b7fda74746a250f29360235ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:13.778975 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4317294e-a2d2-450c-b809-55889b51065a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:13.847843 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1413/2838] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:09:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:13.861193 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f059b1-193d-40f1-9b83-455b337527f3 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:13.862848 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-70f059b1-193d-40f1-9b83-455b337527f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:13.922283 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f059b1-193d-40f1-9b83-455b337527f3 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:13.925927 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49212020-7e1e-4426-9f8d-a6f957c84bde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:13.926534 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49212020-7e1e-4426-9f8d-a6f957c84bde service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:13.926814 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49212020-7e1e-4426-9f8d-a6f957c84bde service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:13.927745 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49212020-7e1e-4426-9f8d-a6f957c84bde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:13.936970 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49212020-7e1e-4426-9f8d-a6f957c84bde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:13.938403 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49212020-7e1e-4426-9f8d-a6f957c84bde service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:13.948029 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1426/2839] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:09:13 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:09:15.129765 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317294e-a2d2-450c-b809-55889b51065a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:15.139309 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317294e-a2d2-450c-b809-55889b51065a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:15.144004 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924afbcf-3695-42ba-916f-e7c06631dd8c tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f65349b7fda74746a250f29360235ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fcce1fcf19bb4820957c5258a5e28126', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:15.144636 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924afbcf-3695-42ba-916f-e7c06631dd8c tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:15.145180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924afbcf-3695-42ba-916f-e7c06631dd8c tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:15.145462 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924afbcf-3695-42ba-916f-e7c06631dd8c tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:15.146657 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924afbcf-3695-42ba-916f-e7c06631dd8c tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:09:15.146967 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924afbcf-3695-42ba-916f-e7c06631dd8c tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:15.148606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924afbcf-3695-42ba-916f-e7c06631dd8c tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:15.160454 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1414/2840] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 15:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:09:15.165236 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49212020-7e1e-4426-9f8d-a6f957c84bde service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:15.174652 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49212020-7e1e-4426-9f8d-a6f957c84bde service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:15.179489 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d391b34-6a9e-4ecb-9870-0afd56917a96 tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f65349b7fda74746a250f29360235ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fcce1fcf19bb4820957c5258a5e28126', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:15.180099 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d391b34-6a9e-4ecb-9870-0afd56917a96 tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:15.180394 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d391b34-6a9e-4ecb-9870-0afd56917a96 tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:15.180675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d391b34-6a9e-4ecb-9870-0afd56917a96 tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:15.181372 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d391b34-6a9e-4ecb-9870-0afd56917a96 tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:15.183296 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d391b34-6a9e-4ecb-9870-0afd56917a96 tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:15.306338 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1427/2841] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 15:09:15 2022] POST /identity/v3/projects => generated 353 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:09:15.310534 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924afbcf-3695-42ba-916f-e7c06631dd8c tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:15.320479 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924afbcf-3695-42ba-916f-e7c06631dd8c tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:15.375845 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18406545-bc14-487b-aa03-50de7097b75e tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f65349b7fda74746a250f29360235ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fcce1fcf19bb4820957c5258a5e28126', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:15.376467 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18406545-bc14-487b-aa03-50de7097b75e tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:15.376763 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18406545-bc14-487b-aa03-50de7097b75e tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:15.377044 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18406545-bc14-487b-aa03-50de7097b75e tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:15.377735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18406545-bc14-487b-aa03-50de7097b75e tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:09:15.378045 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18406545-bc14-487b-aa03-50de7097b75e tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:15.379671 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18406545-bc14-487b-aa03-50de7097b75e tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:15.393076 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1415/2842] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 15:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:09:15.431850 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d391b34-6a9e-4ecb-9870-0afd56917a96 tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:15.442185 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d391b34-6a9e-4ecb-9870-0afd56917a96 tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:15.447618 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca106f02-5800-4fa0-953f-f35daefb53bb tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f65349b7fda74746a250f29360235ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fcce1fcf19bb4820957c5258a5e28126', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:15.448260 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca106f02-5800-4fa0-953f-f35daefb53bb tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:15.448543 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca106f02-5800-4fa0-953f-f35daefb53bb tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:15.448853 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca106f02-5800-4fa0-953f-f35daefb53bb tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] PATH_INFO: `/v3/projects/76910062ce754fcfa4c48b2aba4c7006` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:15.450012 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca106f02-5800-4fa0-953f-f35daefb53bb tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] RBAC: Authorizing `identity:delete_project(project_id=76910062ce754fcfa4c48b2aba4c7006)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:15.451972 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca106f02-5800-4fa0-953f-f35daefb53bb tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:15.640382 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=92,keystone:INSERT=6,keystone:DELETE=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:15.652624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ca106f02-5800-4fa0-953f-f35daefb53bb tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76910062ce754fcfa4c48b2aba4c7006', 'request_id': 'req-ca106f02-5800-4fa0-953f-f35daefb53bb'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:15.658116 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ca106f02-5800-4fa0-953f-f35daefb53bb tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76910062ce754fcfa4c48b2aba4c7006', 'request_id': 'req-ca106f02-5800-4fa0-953f-f35daefb53bb'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:15.767566 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ca106f02-5800-4fa0-953f-f35daefb53bb tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76910062ce754fcfa4c48b2aba4c7006', 'request_id': 'req-ca106f02-5800-4fa0-953f-f35daefb53bb'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:15.780488 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ca106f02-5800-4fa0-953f-f35daefb53bb tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] The token cache is being invalidate because project 76910062ce754fcfa4c48b2aba4c7006 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:09:15.780778 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ca106f02-5800-4fa0-953f-f35daefb53bb tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:15.781432 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ca106f02-5800-4fa0-953f-f35daefb53bb tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:15.782966 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1428/2843] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:09:15 2022] DELETE /identity/v3/projects/76910062ce754fcfa4c48b2aba4c7006 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:16.594269 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18406545-bc14-487b-aa03-50de7097b75e tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:16.596324 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-18406545-bc14-487b-aa03-50de7097b75e tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:16.652356 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:16.668996 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18406545-bc14-487b-aa03-50de7097b75e tempest-VolumeQuotaClassesTest-1697517679 tempest-VolumeQuotaClassesTest-1697517679-project-admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:16.722652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c96239-432d-4c5b-a7c2-697d20c2ea25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:16.723290 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c96239-432d-4c5b-a7c2-697d20c2ea25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:16.723605 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c96239-432d-4c5b-a7c2-697d20c2ea25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:16.724618 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c96239-432d-4c5b-a7c2-697d20c2ea25 admin admin] PATH_INFO: `/v3/users/63c08b471ad84c5f8ca0b8b88e96b8f6` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:16.730935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c96239-432d-4c5b-a7c2-697d20c2ea25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63c08b471ad84c5f8ca0b8b88e96b8f6)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:16.730935 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c96239-432d-4c5b-a7c2-697d20c2ea25 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:16.978921 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d2c96239-432d-4c5b-a7c2-697d20c2ea25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63c08b471ad84c5f8ca0b8b88e96b8f6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:16.979531 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d2c96239-432d-4c5b-a7c2-697d20c2ea25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63c08b471ad84c5f8ca0b8b88e96b8f6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:16.991528 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d2c96239-432d-4c5b-a7c2-697d20c2ea25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63c08b471ad84c5f8ca0b8b88e96b8f6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:16.998252 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d2c96239-432d-4c5b-a7c2-697d20c2ea25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63c08b471ad84c5f8ca0b8b88e96b8f6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:16.998888 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d2c96239-432d-4c5b-a7c2-697d20c2ea25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63c08b471ad84c5f8ca0b8b88e96b8f6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:17.105139 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1416/2844] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:09:16 2022] DELETE /identity/v3/users/63c08b471ad84c5f8ca0b8b88e96b8f6 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:17.109911 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae9037-33e5-408f-98d0-fefbf68f2d41 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:17.110391 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae9037-33e5-408f-98d0-fefbf68f2d41 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:17.110546 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae9037-33e5-408f-98d0-fefbf68f2d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:17.129803 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-92ae9037-33e5-408f-98d0-fefbf68f2d41 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:17.132042 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-92ae9037-33e5-408f-98d0-fefbf68f2d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:17.194435 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1429/2845] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:09:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:17.204199 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c96239-432d-4c5b-a7c2-697d20c2ea25 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:17.205606 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d2c96239-432d-4c5b-a7c2-697d20c2ea25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:17.263268 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c96239-432d-4c5b-a7c2-697d20c2ea25 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:17.266906 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d5119c-cf89-465d-aa6d-7ccdb74fbe71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:17.267558 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5119c-cf89-465d-aa6d-7ccdb74fbe71 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:17.267855 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5119c-cf89-465d-aa6d-7ccdb74fbe71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:17.268121 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5119c-cf89-465d-aa6d-7ccdb74fbe71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:17.277466 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d5119c-cf89-465d-aa6d-7ccdb74fbe71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:17.278898 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d5119c-cf89-465d-aa6d-7ccdb74fbe71 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:17.288418 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1417/2846] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:09:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:09:17.771641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ae9037-33e5-408f-98d0-fefbf68f2d41 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:17.778166 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-92ae9037-33e5-408f-98d0-fefbf68f2d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:17.786309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ae9037-33e5-408f-98d0-fefbf68f2d41 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:17.791020 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ef81f-e918-43ce-9003-12fd13ea9232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:17.792080 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ef81f-e918-43ce-9003-12fd13ea9232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:17.792080 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ef81f-e918-43ce-9003-12fd13ea9232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:17.792353 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ef81f-e918-43ce-9003-12fd13ea9232 admin admin] PATH_INFO: `/v3/projects/dbb6c9f63b1848f3a480e7771e79a157` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:17.793542 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ef81f-e918-43ce-9003-12fd13ea9232 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbb6c9f63b1848f3a480e7771e79a157)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:17.796431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ef81f-e918-43ce-9003-12fd13ea9232 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:17.999100 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3e9ef81f-e918-43ce-9003-12fd13ea9232 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbb6c9f63b1848f3a480e7771e79a157', 'request_id': 'req-3e9ef81f-e918-43ce-9003-12fd13ea9232'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:18.004845 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3e9ef81f-e918-43ce-9003-12fd13ea9232 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbb6c9f63b1848f3a480e7771e79a157', 'request_id': 'req-3e9ef81f-e918-43ce-9003-12fd13ea9232'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:18.162926 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3e9ef81f-e918-43ce-9003-12fd13ea9232 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbb6c9f63b1848f3a480e7771e79a157', 'request_id': 'req-3e9ef81f-e918-43ce-9003-12fd13ea9232'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:18.175010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3e9ef81f-e918-43ce-9003-12fd13ea9232 admin admin] The token cache is being invalidate because project dbb6c9f63b1848f3a480e7771e79a157 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:09:18.175300 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3e9ef81f-e918-43ce-9003-12fd13ea9232 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:18.175987 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3e9ef81f-e918-43ce-9003-12fd13ea9232 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:18.177613 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1430/2847] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:09:17 2022] DELETE /identity/v3/projects/dbb6c9f63b1848f3a480e7771e79a157 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:09:18.181830 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d5119c-cf89-465d-aa6d-7ccdb74fbe71 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:18.191533 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d5119c-cf89-465d-aa6d-7ccdb74fbe71 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:18.200359 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:18.248180 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8070fc9-633c-443a-beb6-87b21a6d97d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:18.248819 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8070fc9-633c-443a-beb6-87b21a6d97d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:18.249105 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8070fc9-633c-443a-beb6-87b21a6d97d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:18.249385 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8070fc9-633c-443a-beb6-87b21a6d97d7 admin admin] PATH_INFO: `/v3/users/f65349b7fda74746a250f29360235ef5` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:18.250597 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8070fc9-633c-443a-beb6-87b21a6d97d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f65349b7fda74746a250f29360235ef5)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:18.252559 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8070fc9-633c-443a-beb6-87b21a6d97d7 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:18.476534 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a8070fc9-633c-443a-beb6-87b21a6d97d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f65349b7fda74746a250f29360235ef5'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:18.477108 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a8070fc9-633c-443a-beb6-87b21a6d97d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f65349b7fda74746a250f29360235ef5'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:18.488853 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a8070fc9-633c-443a-beb6-87b21a6d97d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f65349b7fda74746a250f29360235ef5'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:18.495651 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a8070fc9-633c-443a-beb6-87b21a6d97d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f65349b7fda74746a250f29360235ef5'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:18.496305 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a8070fc9-633c-443a-beb6-87b21a6d97d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f65349b7fda74746a250f29360235ef5'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:18.599614 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1418/2848] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:09:18 2022] DELETE /identity/v3/users/f65349b7fda74746a250f29360235ef5 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:19.207857 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ef81f-e918-43ce-9003-12fd13ea9232 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:19.209408 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3e9ef81f-e918-43ce-9003-12fd13ea9232 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:19.222951 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ef81f-e918-43ce-9003-12fd13ea9232 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:19.267377 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:19.282896 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ab46f-c298-46a4-a42c-e3b1a0f91c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:19.283578 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ab46f-c298-46a4-a42c-e3b1a0f91c97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:19.283861 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ab46f-c298-46a4-a42c-e3b1a0f91c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:19.284140 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ab46f-c298-46a4-a42c-e3b1a0f91c97 admin admin] PATH_INFO: `/v3/projects/fcce1fcf19bb4820957c5258a5e28126` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:19.285318 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ab46f-c298-46a4-a42c-e3b1a0f91c97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcce1fcf19bb4820957c5258a5e28126)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:19.287878 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ab46f-c298-46a4-a42c-e3b1a0f91c97 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:19.464991 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6f6ab46f-c298-46a4-a42c-e3b1a0f91c97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcce1fcf19bb4820957c5258a5e28126', 'request_id': 'req-6f6ab46f-c298-46a4-a42c-e3b1a0f91c97'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:19.470332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6f6ab46f-c298-46a4-a42c-e3b1a0f91c97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcce1fcf19bb4820957c5258a5e28126', 'request_id': 'req-6f6ab46f-c298-46a4-a42c-e3b1a0f91c97'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:19.596629 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6f6ab46f-c298-46a4-a42c-e3b1a0f91c97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcce1fcf19bb4820957c5258a5e28126', 'request_id': 'req-6f6ab46f-c298-46a4-a42c-e3b1a0f91c97'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:19.609299 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6f6ab46f-c298-46a4-a42c-e3b1a0f91c97 admin admin] The token cache is being invalidate because project fcce1fcf19bb4820957c5258a5e28126 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:09:19.609615 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6f6ab46f-c298-46a4-a42c-e3b1a0f91c97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:19.610445 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6f6ab46f-c298-46a4-a42c-e3b1a0f91c97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:19.612044 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1431/2849] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:09:19 2022] DELETE /identity/v3/projects/fcce1fcf19bb4820957c5258a5e28126 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:19.617898 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a1c90-8932-475d-b20c-071ea6df8162 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:19.618407 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a1c90-8932-475d-b20c-071ea6df8162 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:19.618529 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a1c90-8932-475d-b20c-071ea6df8162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:19.638104 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-0a9a1c90-8932-475d-b20c-071ea6df8162 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:19.639329 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0a9a1c90-8932-475d-b20c-071ea6df8162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:19.700470 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1419/2850] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:19.709035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d0a80-638a-4ef1-961c-97810a22223f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:19.709328 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d0a80-638a-4ef1-961c-97810a22223f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:19.709610 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d0a80-638a-4ef1-961c-97810a22223f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:19.730291 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ee0d0a80-638a-4ef1-961c-97810a22223f None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:19.731537 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ee0d0a80-638a-4ef1-961c-97810a22223f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:19.740210 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1432/2851] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:19.744701 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a1c90-8932-475d-b20c-071ea6df8162 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:19.754500 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a1c90-8932-475d-b20c-071ea6df8162 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:19.759897 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d9c2b-ecb8-4dd6-964e-7a26917d7881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:19.760513 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d9c2b-ecb8-4dd6-964e-7a26917d7881 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:19.760799 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d9c2b-ecb8-4dd6-964e-7a26917d7881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:19.761080 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d9c2b-ecb8-4dd6-964e-7a26917d7881 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:19.761786 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d9c2b-ecb8-4dd6-964e-7a26917d7881 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:09:19.762098 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d9c2b-ecb8-4dd6-964e-7a26917d7881 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:19.763735 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d9c2b-ecb8-4dd6-964e-7a26917d7881 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:19.776585 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1420/2852] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 15:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:09:19.781061 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d0a80-638a-4ef1-961c-97810a22223f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:19.790646 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d0a80-638a-4ef1-961c-97810a22223f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:19.795592 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336cfc27-4c8e-4d23-bdda-5450d5c92661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:19.796203 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cfc27-4c8e-4d23-bdda-5450d5c92661 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:19.796485 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cfc27-4c8e-4d23-bdda-5450d5c92661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:19.796762 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cfc27-4c8e-4d23-bdda-5450d5c92661 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:19.797422 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336cfc27-4c8e-4d23-bdda-5450d5c92661 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:19.799350 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336cfc27-4c8e-4d23-bdda-5450d5c92661 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:19.926939 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1433/2853] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 15:09:19 2022] POST /identity/v3/projects => generated 375 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:09:19.931725 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d9c2b-ecb8-4dd6-964e-7a26917d7881 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:19.941881 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d9c2b-ecb8-4dd6-964e-7a26917d7881 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:19.959949 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:20.000103 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e968a4e-c5d8-4f5d-a97a-32a211af856b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:20.000648 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e968a4e-c5d8-4f5d-a97a-32a211af856b admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:20.001017 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e968a4e-c5d8-4f5d-a97a-32a211af856b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:20.001493 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e968a4e-c5d8-4f5d-a97a-32a211af856b admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:20.002669 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e968a4e-c5d8-4f5d-a97a-32a211af856b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:20.004678 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e968a4e-c5d8-4f5d-a97a-32a211af856b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:20.162893 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1421/2854] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 15:09:19 2022] POST /identity/v3/users => generated 288 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:09:20.167106 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336cfc27-4c8e-4d23-bdda-5450d5c92661 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:20.177233 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336cfc27-4c8e-4d23-bdda-5450d5c92661 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:20.181740 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec59e6-d466-450f-9c70-ebe36b149319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:20.182463 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec59e6-d466-450f-9c70-ebe36b149319 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:20.182752 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec59e6-d466-450f-9c70-ebe36b149319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:20.183028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec59e6-d466-450f-9c70-ebe36b149319 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:20.183790 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec59e6-d466-450f-9c70-ebe36b149319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:20.185429 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec59e6-d466-450f-9c70-ebe36b149319 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:20.196007 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1434/2855] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:09:20.199851 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e968a4e-c5d8-4f5d-a97a-32a211af856b admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:20.210727 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e968a4e-c5d8-4f5d-a97a-32a211af856b admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:20.215427 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59010559-10fa-42f4-8abb-b8492cd28c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:20.216048 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59010559-10fa-42f4-8abb-b8492cd28c68 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:20.216336 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59010559-10fa-42f4-8abb-b8492cd28c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:20.216616 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59010559-10fa-42f4-8abb-b8492cd28c68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:20.217318 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59010559-10fa-42f4-8abb-b8492cd28c68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:20.219030 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59010559-10fa-42f4-8abb-b8492cd28c68 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:20.230416 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1422/2856] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:09:20.234639 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec59e6-d466-450f-9c70-ebe36b149319 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:20.245403 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec59e6-d466-450f-9c70-ebe36b149319 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:20.250078 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413c5e5-fa8b-4388-954d-4bab77d1e028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:20.250781 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413c5e5-fa8b-4388-954d-4bab77d1e028 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:20.251024 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413c5e5-fa8b-4388-954d-4bab77d1e028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:20.251299 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413c5e5-fa8b-4388-954d-4bab77d1e028 admin admin] PATH_INFO: `/v3/projects/c422e15e621f4466aad24fe5b49d0cac/users/3c1bc2d9d3a54ab485687e3d9ff2a5b9/roles/e033961a9363456a8510c34414dd839e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:20.266719 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1413c5e5-fa8b-4388-954d-4bab77d1e028 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c422e15e621f4466aad24fe5b49d0cac, user_id=3c1bc2d9d3a54ab485687e3d9ff2a5b9, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:20.269408 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1413c5e5-fa8b-4388-954d-4bab77d1e028 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:20.357005 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1435/2857] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:09:20 2022] PUT /identity/v3/projects/c422e15e621f4466aad24fe5b49d0cac/users/3c1bc2d9d3a54ab485687e3d9ff2a5b9/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:20.370021 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff71872-3b27-4748-82ce-5f948d384c6d None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:20.370427 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff71872-3b27-4748-82ce-5f948d384c6d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:20.370713 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff71872-3b27-4748-82ce-5f948d384c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:20.400059 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3ff71872-3b27-4748-82ce-5f948d384c6d None None] MFA Rules not processed for user `3c1bc2d9d3a54ab485687e3d9ff2a5b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:20.401335 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3ff71872-3b27-4748-82ce-5f948d384c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:20.469747 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1423/2858] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:09:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:20.474506 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413c5e5-fa8b-4388-954d-4bab77d1e028 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:20.539356 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413c5e5-fa8b-4388-954d-4bab77d1e028 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:20.544298 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb8677-223f-49e4-98f7-26c450e7fc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:20.544906 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb8677-223f-49e4-98f7-26c450e7fc72 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:20.545354 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb8677-223f-49e4-98f7-26c450e7fc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:20.546598 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb8677-223f-49e4-98f7-26c450e7fc72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:20.546598 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb8677-223f-49e4-98f7-26c450e7fc72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:20.548055 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb8677-223f-49e4-98f7-26c450e7fc72 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:20.564600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:20.707140 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1436/2859] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 15:09:20 2022] POST /identity/v3/projects => generated 375 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:09:20.711945 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff71872-3b27-4748-82ce-5f948d384c6d None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:20.722730 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff71872-3b27-4748-82ce-5f948d384c6d None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:20.784892 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7df6e-2a7a-4b35-be9f-991e011b1f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:20.785520 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7df6e-2a7a-4b35-be9f-991e011b1f91 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:20.785803 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7df6e-2a7a-4b35-be9f-991e011b1f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:20.786079 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7df6e-2a7a-4b35-be9f-991e011b1f91 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:20.786937 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b7df6e-2a7a-4b35-be9f-991e011b1f91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:20.788893 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b7df6e-2a7a-4b35-be9f-991e011b1f91 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:20.942925 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1424/2860] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 15:09:20 2022] POST /identity/v3/users => generated 288 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:09:20.946972 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb8677-223f-49e4-98f7-26c450e7fc72 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:20.957677 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb8677-223f-49e4-98f7-26c450e7fc72 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:20.962226 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ac475-aacb-49d8-b2c6-9b6c47d81c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:20.962847 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ac475-aacb-49d8-b2c6-9b6c47d81c23 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:20.963148 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ac475-aacb-49d8-b2c6-9b6c47d81c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:20.963447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ac475-aacb-49d8-b2c6-9b6c47d81c23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:20.964152 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ac475-aacb-49d8-b2c6-9b6c47d81c23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:20.965788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ac475-aacb-49d8-b2c6-9b6c47d81c23 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:20.977121 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1437/2861] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:09:20.981484 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7df6e-2a7a-4b35-be9f-991e011b1f91 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:20.992544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7df6e-2a7a-4b35-be9f-991e011b1f91 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:20.997204 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f06bcb-4c95-49a7-880a-c53e4f4f57a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:20.997818 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f06bcb-4c95-49a7-880a-c53e4f4f57a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:20.998103 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f06bcb-4c95-49a7-880a-c53e4f4f57a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:20.998401 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f06bcb-4c95-49a7-880a-c53e4f4f57a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:20.999228 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f06bcb-4c95-49a7-880a-c53e4f4f57a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:21.000847 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f06bcb-4c95-49a7-880a-c53e4f4f57a4 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:21.011492 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1425/2862] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:09:21.015448 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ac475-aacb-49d8-b2c6-9b6c47d81c23 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:21.026579 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ac475-aacb-49d8-b2c6-9b6c47d81c23 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:21.031441 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f723dcf-070c-4a42-9ce3-f6fe196e922f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:21.032109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f723dcf-070c-4a42-9ce3-f6fe196e922f admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:21.032399 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f723dcf-070c-4a42-9ce3-f6fe196e922f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:21.032683 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f723dcf-070c-4a42-9ce3-f6fe196e922f admin admin] PATH_INFO: `/v3/projects/0afe620d095a4ff29661e4358625e33e/users/630bd052bd4c41438b9767595bb669e5/roles/e033961a9363456a8510c34414dd839e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:21.049595 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f723dcf-070c-4a42-9ce3-f6fe196e922f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0afe620d095a4ff29661e4358625e33e, user_id=630bd052bd4c41438b9767595bb669e5, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:21.052811 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f723dcf-070c-4a42-9ce3-f6fe196e922f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:21.139269 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1438/2863] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:09:21 2022] PUT /identity/v3/projects/0afe620d095a4ff29661e4358625e33e/users/630bd052bd4c41438b9767595bb669e5/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:21.160115 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e11c2-6236-4f33-bbe1-e68daec74347 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:21.160569 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e11c2-6236-4f33-bbe1-e68daec74347 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:21.160982 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e11c2-6236-4f33-bbe1-e68daec74347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:21.176980 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:21.191845 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b68e11c2-6236-4f33-bbe1-e68daec74347 None None] MFA Rules not processed for user `630bd052bd4c41438b9767595bb669e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:21.193338 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b68e11c2-6236-4f33-bbe1-e68daec74347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:21.257352 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1426/2864] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:09:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:21.261906 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f723dcf-070c-4a42-9ce3-f6fe196e922f admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:21.328680 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f723dcf-070c-4a42-9ce3-f6fe196e922f admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:21.336431 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497747f-bc5c-415f-b3dd-1dcb3188579b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:21.337049 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497747f-bc5c-415f-b3dd-1dcb3188579b admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:21.337792 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497747f-bc5c-415f-b3dd-1dcb3188579b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:21.338106 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497747f-bc5c-415f-b3dd-1dcb3188579b admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:21.339881 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4497747f-bc5c-415f-b3dd-1dcb3188579b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:21.341955 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4497747f-bc5c-415f-b3dd-1dcb3188579b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:21.450397 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1439/2865] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 15:09:21 2022] POST /identity/v3/projects => generated 377 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:09:21.454539 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e11c2-6236-4f33-bbe1-e68daec74347 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:21.466064 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e11c2-6236-4f33-bbe1-e68daec74347 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:21.521618 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f56f22d-4d47-47e1-8a1e-a678abfb0126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:21.522227 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f56f22d-4d47-47e1-8a1e-a678abfb0126 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:21.522506 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f56f22d-4d47-47e1-8a1e-a678abfb0126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:21.522773 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f56f22d-4d47-47e1-8a1e-a678abfb0126 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:21.523456 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f56f22d-4d47-47e1-8a1e-a678abfb0126 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:21.525369 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f56f22d-4d47-47e1-8a1e-a678abfb0126 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:21.625072 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1427/2866] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 15:09:21 2022] POST /identity/v3/users => generated 295 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:09:21.629052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497747f-bc5c-415f-b3dd-1dcb3188579b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:21.640035 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497747f-bc5c-415f-b3dd-1dcb3188579b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:21.651422 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f5958-05ea-40ab-abb4-15b6103663a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:21.651422 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f5958-05ea-40ab-abb4-15b6103663a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:21.651422 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f5958-05ea-40ab-abb4-15b6103663a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:21.651422 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f5958-05ea-40ab-abb4-15b6103663a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:21.651856 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f5958-05ea-40ab-abb4-15b6103663a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:21.651856 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f5958-05ea-40ab-abb4-15b6103663a0 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:21.659332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1440/2867] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:09:21.663630 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f56f22d-4d47-47e1-8a1e-a678abfb0126 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:21.674302 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f56f22d-4d47-47e1-8a1e-a678abfb0126 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:21.678962 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1130fbe-6f51-47f2-b9dc-26f756c7e999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:21.679652 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1130fbe-6f51-47f2-b9dc-26f756c7e999 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:21.679955 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1130fbe-6f51-47f2-b9dc-26f756c7e999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:21.680218 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1130fbe-6f51-47f2-b9dc-26f756c7e999 admin admin] PATH_INFO: `/v3/projects/df3c100a085c40f4a927282ed8efe3b8/users/648f968f76514023b0ac8e36c3c6e0d2/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:21.695475 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1130fbe-6f51-47f2-b9dc-26f756c7e999 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df3c100a085c40f4a927282ed8efe3b8, user_id=648f968f76514023b0ac8e36c3c6e0d2, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:21.698689 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1130fbe-6f51-47f2-b9dc-26f756c7e999 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:21.836696 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1428/2868] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:09:21 2022] PUT /identity/v3/projects/df3c100a085c40f4a927282ed8efe3b8/users/648f968f76514023b0ac8e36c3c6e0d2/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:21.840817 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f5958-05ea-40ab-abb4-15b6103663a0 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:21.851157 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f5958-05ea-40ab-abb4-15b6103663a0 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:21.909809 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e501fda-a950-447e-904b-1fd336977a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:21.910460 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e501fda-a950-447e-904b-1fd336977a3c admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:21.910744 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e501fda-a950-447e-904b-1fd336977a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:21.911026 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e501fda-a950-447e-904b-1fd336977a3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:21.911961 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e501fda-a950-447e-904b-1fd336977a3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:21.913597 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e501fda-a950-447e-904b-1fd336977a3c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:21.926701 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1441/2869] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:09:21.930743 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1130fbe-6f51-47f2-b9dc-26f756c7e999 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:21.941844 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1130fbe-6f51-47f2-b9dc-26f756c7e999 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:21.949334 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0399f5f4-357b-49d7-a32a-43861eafe3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:21.949334 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0399f5f4-357b-49d7-a32a-43861eafe3ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:21.949334 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0399f5f4-357b-49d7-a32a-43861eafe3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:21.949334 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0399f5f4-357b-49d7-a32a-43861eafe3ed admin admin] PATH_INFO: `/v3/projects/df3c100a085c40f4a927282ed8efe3b8/users/648f968f76514023b0ac8e36c3c6e0d2/roles/51b2d10dfffb4b75b186f3d8b15e2a78` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:21.950488 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0399f5f4-357b-49d7-a32a-43861eafe3ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=df3c100a085c40f4a927282ed8efe3b8, user_id=648f968f76514023b0ac8e36c3c6e0d2, role_id=51b2d10dfffb4b75b186f3d8b15e2a78)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:21.953375 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0399f5f4-357b-49d7-a32a-43861eafe3ed admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:21.963875 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1429/2870] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:09:21 2022] PUT /identity/v3/projects/df3c100a085c40f4a927282ed8efe3b8/users/648f968f76514023b0ac8e36c3c6e0d2/roles/51b2d10dfffb4b75b186f3d8b15e2a78 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:21.974138 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a3ad7-3ab8-4620-ad5e-4c62b20e8da3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:21.974426 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a3ad7-3ab8-4620-ad5e-4c62b20e8da3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:21.974709 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a3ad7-3ab8-4620-ad5e-4c62b20e8da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:22.002625 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-125a3ad7-3ab8-4620-ad5e-4c62b20e8da3 None None] MFA Rules not processed for user `648f968f76514023b0ac8e36c3c6e0d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:22.005857 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-125a3ad7-3ab8-4620-ad5e-4c62b20e8da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:22.023841 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:22.075027 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1442/2871] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:09:21 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:22.087266 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0399f5f4-357b-49d7-a32a-43861eafe3ed admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:22.089026 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0399f5f4-357b-49d7-a32a-43861eafe3ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:22.145458 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:22.149238 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0399f5f4-357b-49d7-a32a-43861eafe3ed admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:22.152953 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4f188-2547-4503-98a1-0a2ef5ac8bfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:22.153690 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4f188-2547-4503-98a1-0a2ef5ac8bfa service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:22.155659 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4f188-2547-4503-98a1-0a2ef5ac8bfa service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:22.155659 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4f188-2547-4503-98a1-0a2ef5ac8bfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:22.210799 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df4f188-2547-4503-98a1-0a2ef5ac8bfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:22.212425 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df4f188-2547-4503-98a1-0a2ef5ac8bfa service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:22.221504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1430/2872] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:09:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:09:25.708030 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a3ad7-3ab8-4620-ad5e-4c62b20e8da3 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:25.718253 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a3ad7-3ab8-4620-ad5e-4c62b20e8da3 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:25.722096 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a79bb-7593-40d9-80ad-57e15d92ee7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:25.722741 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a79bb-7593-40d9-80ad-57e15d92ee7c service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:25.723023 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a79bb-7593-40d9-80ad-57e15d92ee7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:25.723298 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a79bb-7593-40d9-80ad-57e15d92ee7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:25.781669 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a79bb-7593-40d9-80ad-57e15d92ee7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:25.783107 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a79bb-7593-40d9-80ad-57e15d92ee7c service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:25.792725 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1443/2873] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:09:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:09:27.572712 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4f188-2547-4503-98a1-0a2ef5ac8bfa service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:27.581279 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4f188-2547-4503-98a1-0a2ef5ac8bfa service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:27.584894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f608839-69d2-48fe-b1ff-face1fe00611 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:27.585530 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f608839-69d2-48fe-b1ff-face1fe00611 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:27.585814 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f608839-69d2-48fe-b1ff-face1fe00611 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:27.586088 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f608839-69d2-48fe-b1ff-face1fe00611 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:27.595343 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f608839-69d2-48fe-b1ff-face1fe00611 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:27.596797 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f608839-69d2-48fe-b1ff-face1fe00611 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:27.607329 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1431/2874] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:09:27 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:09:32.326331 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a79bb-7593-40d9-80ad-57e15d92ee7c service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:32.390604 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a79bb-7593-40d9-80ad-57e15d92ee7c service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:32.396264 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443dde9-6704-41fc-97df-2ba1b984b624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:32.396924 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443dde9-6704-41fc-97df-2ba1b984b624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:32.397225 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443dde9-6704-41fc-97df-2ba1b984b624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:32.397581 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443dde9-6704-41fc-97df-2ba1b984b624 admin admin] PATH_INFO: `/v3/users/3c1bc2d9d3a54ab485687e3d9ff2a5b9` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:32.398796 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c443dde9-6704-41fc-97df-2ba1b984b624 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c1bc2d9d3a54ab485687e3d9ff2a5b9)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:32.401149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c443dde9-6704-41fc-97df-2ba1b984b624 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:32.749615 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c443dde9-6704-41fc-97df-2ba1b984b624 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c1bc2d9d3a54ab485687e3d9ff2a5b9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:32.750309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c443dde9-6704-41fc-97df-2ba1b984b624 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c1bc2d9d3a54ab485687e3d9ff2a5b9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:32.758980 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:32.764466 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c443dde9-6704-41fc-97df-2ba1b984b624 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c1bc2d9d3a54ab485687e3d9ff2a5b9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:32.772428 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c443dde9-6704-41fc-97df-2ba1b984b624 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c1bc2d9d3a54ab485687e3d9ff2a5b9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:32.773184 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c443dde9-6704-41fc-97df-2ba1b984b624 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c1bc2d9d3a54ab485687e3d9ff2a5b9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:32.920553 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1444/2875] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:09:32 2022] DELETE /identity/v3/users/3c1bc2d9d3a54ab485687e3d9ff2a5b9 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:32.925241 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b01e2-79be-420c-923f-77cc2cb05575 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:32.925760 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b01e2-79be-420c-923f-77cc2cb05575 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:32.925877 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b01e2-79be-420c-923f-77cc2cb05575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:32.945429 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-642b01e2-79be-420c-923f-77cc2cb05575 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:32.946693 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-642b01e2-79be-420c-923f-77cc2cb05575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:33.007695 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1432/2876] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:09:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:33.017311 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443dde9-6704-41fc-97df-2ba1b984b624 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:33.018774 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c443dde9-6704-41fc-97df-2ba1b984b624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:33.081658 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443dde9-6704-41fc-97df-2ba1b984b624 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:33.085337 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a7a61-0c04-447a-881e-f51716baa9f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:33.086158 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a7a61-0c04-447a-881e-f51716baa9f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:33.086322 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a7a61-0c04-447a-881e-f51716baa9f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:33.086503 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a7a61-0c04-447a-881e-f51716baa9f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:33.096481 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a7a61-0c04-447a-881e-f51716baa9f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:33.097950 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a7a61-0c04-447a-881e-f51716baa9f1 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:33.108502 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1445/2877] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:09:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:09:33.557795 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b01e2-79be-420c-923f-77cc2cb05575 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:33.557795 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-642b01e2-79be-420c-923f-77cc2cb05575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:33.565582 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b01e2-79be-420c-923f-77cc2cb05575 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:33.570564 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ddfa0-eddf-4ac7-a34f-d79aadcee6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:33.571258 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ddfa0-eddf-4ac7-a34f-d79aadcee6ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:33.571615 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ddfa0-eddf-4ac7-a34f-d79aadcee6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:33.571894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ddfa0-eddf-4ac7-a34f-d79aadcee6ae admin admin] PATH_INFO: `/v3/projects/c422e15e621f4466aad24fe5b49d0cac` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:33.573291 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ddfa0-eddf-4ac7-a34f-d79aadcee6ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=c422e15e621f4466aad24fe5b49d0cac)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:33.575073 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ddfa0-eddf-4ac7-a34f-d79aadcee6ae admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:33.798131 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f17ddfa0-eddf-4ac7-a34f-d79aadcee6ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c422e15e621f4466aad24fe5b49d0cac', 'request_id': 'req-f17ddfa0-eddf-4ac7-a34f-d79aadcee6ae'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:33.804660 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f17ddfa0-eddf-4ac7-a34f-d79aadcee6ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c422e15e621f4466aad24fe5b49d0cac', 'request_id': 'req-f17ddfa0-eddf-4ac7-a34f-d79aadcee6ae'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:33.956883 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f17ddfa0-eddf-4ac7-a34f-d79aadcee6ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c422e15e621f4466aad24fe5b49d0cac', 'request_id': 'req-f17ddfa0-eddf-4ac7-a34f-d79aadcee6ae'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:33.969379 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f17ddfa0-eddf-4ac7-a34f-d79aadcee6ae admin admin] The token cache is being invalidate because project c422e15e621f4466aad24fe5b49d0cac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:09:33.972603 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f17ddfa0-eddf-4ac7-a34f-d79aadcee6ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:33.972603 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f17ddfa0-eddf-4ac7-a34f-d79aadcee6ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:33.972603 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1433/2878] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:09:33 2022] DELETE /identity/v3/projects/c422e15e621f4466aad24fe5b49d0cac => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:33.976800 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a7a61-0c04-447a-881e-f51716baa9f1 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:33.986485 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a7a61-0c04-447a-881e-f51716baa9f1 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:34.045905 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a3576-23da-4ccb-9a05-45ae82cebf62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:34.046587 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a3576-23da-4ccb-9a05-45ae82cebf62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:34.046874 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a3576-23da-4ccb-9a05-45ae82cebf62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:34.047184 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a3576-23da-4ccb-9a05-45ae82cebf62 admin admin] PATH_INFO: `/v3/users/630bd052bd4c41438b9767595bb669e5` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:34.048492 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7a3576-23da-4ccb-9a05-45ae82cebf62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=630bd052bd4c41438b9767595bb669e5)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:34.050365 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7a3576-23da-4ccb-9a05-45ae82cebf62 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:34.353788 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3d7a3576-23da-4ccb-9a05-45ae82cebf62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '630bd052bd4c41438b9767595bb669e5'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:34.354550 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3d7a3576-23da-4ccb-9a05-45ae82cebf62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '630bd052bd4c41438b9767595bb669e5'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:34.366956 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3d7a3576-23da-4ccb-9a05-45ae82cebf62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '630bd052bd4c41438b9767595bb669e5'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:34.373970 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3d7a3576-23da-4ccb-9a05-45ae82cebf62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '630bd052bd4c41438b9767595bb669e5'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:34.374675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3d7a3576-23da-4ccb-9a05-45ae82cebf62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '630bd052bd4c41438b9767595bb669e5'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:34.378601 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:34.481248 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1446/2879] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:09:33 2022] DELETE /identity/v3/users/630bd052bd4c41438b9767595bb669e5 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:35.098506 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ddfa0-eddf-4ac7-a34f-d79aadcee6ae admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:35.100659 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f17ddfa0-eddf-4ac7-a34f-d79aadcee6ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:35.113921 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ddfa0-eddf-4ac7-a34f-d79aadcee6ae admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:35.126064 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:35.175516 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0256cfab-8fd6-4c71-b0f4-3554839e7eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:35.176166 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256cfab-8fd6-4c71-b0f4-3554839e7eb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:35.176451 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256cfab-8fd6-4c71-b0f4-3554839e7eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:35.176731 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256cfab-8fd6-4c71-b0f4-3554839e7eb7 admin admin] PATH_INFO: `/v3/projects/0afe620d095a4ff29661e4358625e33e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:35.178480 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0256cfab-8fd6-4c71-b0f4-3554839e7eb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0afe620d095a4ff29661e4358625e33e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:35.180458 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0256cfab-8fd6-4c71-b0f4-3554839e7eb7 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:35.346931 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0256cfab-8fd6-4c71-b0f4-3554839e7eb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0afe620d095a4ff29661e4358625e33e', 'request_id': 'req-0256cfab-8fd6-4c71-b0f4-3554839e7eb7'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:35.352354 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0256cfab-8fd6-4c71-b0f4-3554839e7eb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0afe620d095a4ff29661e4358625e33e', 'request_id': 'req-0256cfab-8fd6-4c71-b0f4-3554839e7eb7'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:35.491748 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0256cfab-8fd6-4c71-b0f4-3554839e7eb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0afe620d095a4ff29661e4358625e33e', 'request_id': 'req-0256cfab-8fd6-4c71-b0f4-3554839e7eb7'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:35.503890 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0256cfab-8fd6-4c71-b0f4-3554839e7eb7 admin admin] The token cache is being invalidate because project 0afe620d095a4ff29661e4358625e33e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:09:35.504198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0256cfab-8fd6-4c71-b0f4-3554839e7eb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:35.504894 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0256cfab-8fd6-4c71-b0f4-3554839e7eb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:35.507050 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1434/2880] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:09:35 2022] DELETE /identity/v3/projects/0afe620d095a4ff29661e4358625e33e => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:35.511213 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a3576-23da-4ccb-9a05-45ae82cebf62 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:35.512692 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3d7a3576-23da-4ccb-9a05-45ae82cebf62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:35.526709 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a3576-23da-4ccb-9a05-45ae82cebf62 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:35.595424 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa298710-c6b1-4de5-9c6c-f9015deffb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:35.596115 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa298710-c6b1-4de5-9c6c-f9015deffb6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:35.596402 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa298710-c6b1-4de5-9c6c-f9015deffb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:35.596689 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa298710-c6b1-4de5-9c6c-f9015deffb6f admin admin] PATH_INFO: `/v3/users/648f968f76514023b0ac8e36c3c6e0d2` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:35.597898 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa298710-c6b1-4de5-9c6c-f9015deffb6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=648f968f76514023b0ac8e36c3c6e0d2)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:35.600532 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa298710-c6b1-4de5-9c6c-f9015deffb6f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:35.894898 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-aa298710-c6b1-4de5-9c6c-f9015deffb6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '648f968f76514023b0ac8e36c3c6e0d2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:35.896658 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-aa298710-c6b1-4de5-9c6c-f9015deffb6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '648f968f76514023b0ac8e36c3c6e0d2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:35.909774 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-aa298710-c6b1-4de5-9c6c-f9015deffb6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '648f968f76514023b0ac8e36c3c6e0d2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:35.916970 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-aa298710-c6b1-4de5-9c6c-f9015deffb6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '648f968f76514023b0ac8e36c3c6e0d2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:35.917689 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-aa298710-c6b1-4de5-9c6c-f9015deffb6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '648f968f76514023b0ac8e36c3c6e0d2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:36.037853 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1447/2881] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:09:35 2022] DELETE /identity/v3/users/648f968f76514023b0ac8e36c3c6e0d2 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:36.701428 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0256cfab-8fd6-4c71-b0f4-3554839e7eb7 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:36.702979 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0256cfab-8fd6-4c71-b0f4-3554839e7eb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:36.716766 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0256cfab-8fd6-4c71-b0f4-3554839e7eb7 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:36.778691 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76eec63-0e43-425b-bf4f-b48ac0f4e57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:36.779267 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76eec63-0e43-425b-bf4f-b48ac0f4e57e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:36.779620 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76eec63-0e43-425b-bf4f-b48ac0f4e57e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:36.779904 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76eec63-0e43-425b-bf4f-b48ac0f4e57e admin admin] PATH_INFO: `/v3/projects/df3c100a085c40f4a927282ed8efe3b8` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:36.781076 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76eec63-0e43-425b-bf4f-b48ac0f4e57e admin admin] RBAC: Authorizing `identity:delete_project(project_id=df3c100a085c40f4a927282ed8efe3b8)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:36.783061 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76eec63-0e43-425b-bf4f-b48ac0f4e57e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:36.924763 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:36.927627 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e76eec63-0e43-425b-bf4f-b48ac0f4e57e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df3c100a085c40f4a927282ed8efe3b8', 'request_id': 'req-e76eec63-0e43-425b-bf4f-b48ac0f4e57e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:36.933078 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e76eec63-0e43-425b-bf4f-b48ac0f4e57e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df3c100a085c40f4a927282ed8efe3b8', 'request_id': 'req-e76eec63-0e43-425b-bf4f-b48ac0f4e57e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:37.019485 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e76eec63-0e43-425b-bf4f-b48ac0f4e57e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df3c100a085c40f4a927282ed8efe3b8', 'request_id': 'req-e76eec63-0e43-425b-bf4f-b48ac0f4e57e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:37.034484 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e76eec63-0e43-425b-bf4f-b48ac0f4e57e admin admin] The token cache is being invalidate because project df3c100a085c40f4a927282ed8efe3b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:09:37.034484 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e76eec63-0e43-425b-bf4f-b48ac0f4e57e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:37.034484 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e76eec63-0e43-425b-bf4f-b48ac0f4e57e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:37.035310 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1435/2882] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:09:36 2022] DELETE /identity/v3/projects/df3c100a085c40f4a927282ed8efe3b8 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:37.041443 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95a680-4ed5-4841-a35f-cf15aafb6f65 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:37.041954 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95a680-4ed5-4841-a35f-cf15aafb6f65 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:37.042165 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95a680-4ed5-4841-a35f-cf15aafb6f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:37.062510 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6f95a680-4ed5-4841-a35f-cf15aafb6f65 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:37.063767 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6f95a680-4ed5-4841-a35f-cf15aafb6f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:37.134473 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1448/2883] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:09:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:09:37.142820 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14224fb8-b972-4591-93d7-73459ea9b75d None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:37.143112 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14224fb8-b972-4591-93d7-73459ea9b75d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:37.143464 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14224fb8-b972-4591-93d7-73459ea9b75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:37.163994 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-14224fb8-b972-4591-93d7-73459ea9b75d None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:37.165260 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-14224fb8-b972-4591-93d7-73459ea9b75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:37.173155 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1436/2884] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:09:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:37.177332 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95a680-4ed5-4841-a35f-cf15aafb6f65 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:37.186999 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95a680-4ed5-4841-a35f-cf15aafb6f65 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:37.191688 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53093095-917c-4bd9-b782-17614210dcea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:37.192292 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53093095-917c-4bd9-b782-17614210dcea admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:37.193153 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53093095-917c-4bd9-b782-17614210dcea admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:37.193432 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53093095-917c-4bd9-b782-17614210dcea admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:37.194122 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53093095-917c-4bd9-b782-17614210dcea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:09:37.194423 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53093095-917c-4bd9-b782-17614210dcea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:37.196028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53093095-917c-4bd9-b782-17614210dcea admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:37.209850 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1449/2885] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 15:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:09:37.214047 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14224fb8-b972-4591-93d7-73459ea9b75d None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:37.223185 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14224fb8-b972-4591-93d7-73459ea9b75d None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:37.228260 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392ee80-6d88-488b-9de2-50c2abc4e72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:37.228871 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392ee80-6d88-488b-9de2-50c2abc4e72a admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:37.229156 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392ee80-6d88-488b-9de2-50c2abc4e72a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:37.229434 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392ee80-6d88-488b-9de2-50c2abc4e72a admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:37.230103 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4392ee80-6d88-488b-9de2-50c2abc4e72a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:37.232066 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4392ee80-6d88-488b-9de2-50c2abc4e72a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:37.338115 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1437/2886] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 15:09:37 2022] POST /identity/v3/projects => generated 369 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:09:37.342621 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53093095-917c-4bd9-b782-17614210dcea admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:37.347863 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:37.357642 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53093095-917c-4bd9-b782-17614210dcea admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:37.420933 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b903a-a684-4f32-805f-3dde23025dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:37.421550 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b903a-a684-4f32-805f-3dde23025dc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:37.421831 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b903a-a684-4f32-805f-3dde23025dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:37.422108 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b903a-a684-4f32-805f-3dde23025dc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:37.423077 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85b903a-a684-4f32-805f-3dde23025dc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:37.425103 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85b903a-a684-4f32-805f-3dde23025dc1 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:37.524871 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1450/2887] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 15:09:37 2022] POST /identity/v3/users => generated 285 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:09:37.528986 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392ee80-6d88-488b-9de2-50c2abc4e72a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:37.539961 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392ee80-6d88-488b-9de2-50c2abc4e72a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:37.544787 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b09054d-c0cc-4838-bb68-516d311b222f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:37.545409 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09054d-c0cc-4838-bb68-516d311b222f admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:37.545692 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09054d-c0cc-4838-bb68-516d311b222f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:37.545969 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09054d-c0cc-4838-bb68-516d311b222f admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:37.546722 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b09054d-c0cc-4838-bb68-516d311b222f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:37.548317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b09054d-c0cc-4838-bb68-516d311b222f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:37.558419 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1438/2888] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:09:37.562612 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b903a-a684-4f32-805f-3dde23025dc1 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:37.573430 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b903a-a684-4f32-805f-3dde23025dc1 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:37.578591 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e1ea8-c826-4c11-9793-4fa0281c1ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:37.579203 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e1ea8-c826-4c11-9793-4fa0281c1ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:37.579580 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e1ea8-c826-4c11-9793-4fa0281c1ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:37.579872 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e1ea8-c826-4c11-9793-4fa0281c1ac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:37.580601 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947e1ea8-c826-4c11-9793-4fa0281c1ac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:37.583010 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947e1ea8-c826-4c11-9793-4fa0281c1ac3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:37.595252 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1451/2889] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:09:37.599777 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b09054d-c0cc-4838-bb68-516d311b222f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:37.611775 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b09054d-c0cc-4838-bb68-516d311b222f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:37.617570 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fadc05-67dd-4f9d-b266-5c41c8cb664b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:37.618405 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fadc05-67dd-4f9d-b266-5c41c8cb664b admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:37.618749 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fadc05-67dd-4f9d-b266-5c41c8cb664b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:37.619118 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fadc05-67dd-4f9d-b266-5c41c8cb664b admin admin] PATH_INFO: `/v3/projects/ff1b71f2838f443fad0b23f34e93dc7d/users/1b8fa1399d254f388a7606965f416921/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:37.634639 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fadc05-67dd-4f9d-b266-5c41c8cb664b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff1b71f2838f443fad0b23f34e93dc7d, user_id=1b8fa1399d254f388a7606965f416921, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:37.637385 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fadc05-67dd-4f9d-b266-5c41c8cb664b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:37.740928 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1439/2890] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:09:37 2022] PUT /identity/v3/projects/ff1b71f2838f443fad0b23f34e93dc7d/users/1b8fa1399d254f388a7606965f416921/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:09:37.753235 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90700698-ed1f-4f00-8c08-30ce1d4c5dcd None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:37.753523 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90700698-ed1f-4f00-8c08-30ce1d4c5dcd None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:37.753802 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90700698-ed1f-4f00-8c08-30ce1d4c5dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:37.781900 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-90700698-ed1f-4f00-8c08-30ce1d4c5dcd None None] MFA Rules not processed for user `1b8fa1399d254f388a7606965f416921`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:37.783113 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-90700698-ed1f-4f00-8c08-30ce1d4c5dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:37.848266 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1452/2891] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:09:37 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:09:37.861315 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fadc05-67dd-4f9d-b266-5c41c8cb664b admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:37.863022 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-24fadc05-67dd-4f9d-b266-5c41c8cb664b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:37.923634 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fadc05-67dd-4f9d-b266-5c41c8cb664b admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:37.927369 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb808044-0805-4b25-b6e2-851aa63fa947 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62adb1f5e9654927847e1717630ff6b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:37.928078 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb808044-0805-4b25-b6e2-851aa63fa947 service glance] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:37.928364 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb808044-0805-4b25-b6e2-851aa63fa947 service glance] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:37.928640 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb808044-0805-4b25-b6e2-851aa63fa947 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:37.937244 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb808044-0805-4b25-b6e2-851aa63fa947 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:37.938673 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb808044-0805-4b25-b6e2-851aa63fa947 service glance] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:37.948140 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1440/2892] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:09:37 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:09:47.844780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=65,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:47.944065 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:DELETE=4,keystone:SELECT=60,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:10:26.885301 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90700698-ed1f-4f00-8c08-30ce1d4c5dcd None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:10:26.990008 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90700698-ed1f-4f00-8c08-30ce1d4c5dcd None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:10:26.994715 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3be015-bc8c-49ec-8f60-666eb0844aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:10:26.996133 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3be015-bc8c-49ec-8f60-666eb0844aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:10:26.996444 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3be015-bc8c-49ec-8f60-666eb0844aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:10:26.996712 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3be015-bc8c-49ec-8f60-666eb0844aee admin admin] PATH_INFO: `/v3/users/1b8fa1399d254f388a7606965f416921` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:10:26.997933 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3be015-bc8c-49ec-8f60-666eb0844aee admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b8fa1399d254f388a7606965f416921)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:10:26.999969 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3be015-bc8c-49ec-8f60-666eb0844aee admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:10:27.245447 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ad3be015-bc8c-49ec-8f60-666eb0844aee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b8fa1399d254f388a7606965f416921'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:10:27.246109 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ad3be015-bc8c-49ec-8f60-666eb0844aee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b8fa1399d254f388a7606965f416921'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:10:27.258675 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ad3be015-bc8c-49ec-8f60-666eb0844aee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b8fa1399d254f388a7606965f416921'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:10:27.265426 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ad3be015-bc8c-49ec-8f60-666eb0844aee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b8fa1399d254f388a7606965f416921'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:10:27.266073 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ad3be015-bc8c-49ec-8f60-666eb0844aee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b8fa1399d254f388a7606965f416921'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:10:27.384013 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1453/2893] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:10:26 2022] DELETE /identity/v3/users/1b8fa1399d254f388a7606965f416921 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:10:27.388663 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15118e-2294-4c0c-a84b-da7670fe96c6 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:10:27.389151 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15118e-2294-4c0c-a84b-da7670fe96c6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:10:27.389264 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15118e-2294-4c0c-a84b-da7670fe96c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:10:27.439631 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-2e15118e-2294-4c0c-a84b-da7670fe96c6 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:10:27.440906 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2e15118e-2294-4c0c-a84b-da7670fe96c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:10:27.499620 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1441/2894] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:10:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:10:27.509597 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3be015-bc8c-49ec-8f60-666eb0844aee admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:10:27.511047 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ad3be015-bc8c-49ec-8f60-666eb0844aee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:10:27.559117 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:10:27.612184 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3be015-bc8c-49ec-8f60-666eb0844aee admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:10:27.617965 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c78d3a0-f31e-466d-bf8a-24fa64b5e05c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:10:27.618720 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78d3a0-f31e-466d-bf8a-24fa64b5e05c service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:10:27.619099 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78d3a0-f31e-466d-bf8a-24fa64b5e05c service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:10:27.619500 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78d3a0-f31e-466d-bf8a-24fa64b5e05c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:10:27.631824 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c78d3a0-f31e-466d-bf8a-24fa64b5e05c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:10:27.633694 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c78d3a0-f31e-466d-bf8a-24fa64b5e05c service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:10:27.646613 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1454/2895] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:10:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:10:28.143198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15118e-2294-4c0c-a84b-da7670fe96c6 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:10:28.144669 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2e15118e-2294-4c0c-a84b-da7670fe96c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:10:28.158095 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15118e-2294-4c0c-a84b-da7670fe96c6 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:10:28.163508 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8d39e-ec61-4a45-b847-24b107147d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:10:28.164295 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8d39e-ec61-4a45-b847-24b107147d3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:10:28.164584 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8d39e-ec61-4a45-b847-24b107147d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:10:28.164864 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8d39e-ec61-4a45-b847-24b107147d3b admin admin] PATH_INFO: `/v3/projects/ff1b71f2838f443fad0b23f34e93dc7d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:10:28.166347 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb8d39e-ec61-4a45-b847-24b107147d3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff1b71f2838f443fad0b23f34e93dc7d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:10:28.168022 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb8d39e-ec61-4a45-b847-24b107147d3b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:10:28.310217 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8cb8d39e-ec61-4a45-b847-24b107147d3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff1b71f2838f443fad0b23f34e93dc7d', 'request_id': 'req-8cb8d39e-ec61-4a45-b847-24b107147d3b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:10:28.316194 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8cb8d39e-ec61-4a45-b847-24b107147d3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff1b71f2838f443fad0b23f34e93dc7d', 'request_id': 'req-8cb8d39e-ec61-4a45-b847-24b107147d3b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:10:28.400674 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8cb8d39e-ec61-4a45-b847-24b107147d3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff1b71f2838f443fad0b23f34e93dc7d', 'request_id': 'req-8cb8d39e-ec61-4a45-b847-24b107147d3b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:10:28.412316 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8cb8d39e-ec61-4a45-b847-24b107147d3b admin admin] The token cache is being invalidate because project ff1b71f2838f443fad0b23f34e93dc7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:10:28.412543 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8cb8d39e-ec61-4a45-b847-24b107147d3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:10:28.413206 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8cb8d39e-ec61-4a45-b847-24b107147d3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:10:28.414729 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1442/2896] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:10:28 2022] DELETE /identity/v3/projects/ff1b71f2838f443fad0b23f34e93dc7d => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:10:28.419775 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f7a2f-88bb-4ffe-886a-ef7dacba0cea None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:10:28.426834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f7a2f-88bb-4ffe-886a-ef7dacba0cea None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:10:28.426834 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f7a2f-88bb-4ffe-886a-ef7dacba0cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:10:28.472077 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-439f7a2f-88bb-4ffe-886a-ef7dacba0cea None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:10:28.473313 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-439f7a2f-88bb-4ffe-886a-ef7dacba0cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:10:28.534238 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1455/2897] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:10:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:10:28.542245 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc797449-b1a5-4f8d-8260-7b773532e859 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:10:28.542636 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc797449-b1a5-4f8d-8260-7b773532e859 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:10:28.543032 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc797449-b1a5-4f8d-8260-7b773532e859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:10:28.564287 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-dc797449-b1a5-4f8d-8260-7b773532e859 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:10:28.565476 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-dc797449-b1a5-4f8d-8260-7b773532e859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:10:28.574103 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1443/2898] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:10:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:10:28.579382 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439f7a2f-88bb-4ffe-886a-ef7dacba0cea None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:10:28.589158 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439f7a2f-88bb-4ffe-886a-ef7dacba0cea None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:10:28.594264 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c64fb-6c5e-477d-b4da-67095c096c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:10:28.594982 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c64fb-6c5e-477d-b4da-67095c096c95 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:10:28.595264 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c64fb-6c5e-477d-b4da-67095c096c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:10:28.595574 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c64fb-6c5e-477d-b4da-67095c096c95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:10:28.596263 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c64fb-6c5e-477d-b4da-67095c096c95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:10:28.596569 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c64fb-6c5e-477d-b4da-67095c096c95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:10:28.598171 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c64fb-6c5e-477d-b4da-67095c096c95 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:10:28.609991 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1456/2899] 10.222.0.97 () {58 vars in 1153 bytes} [Tue Jul 26 15:10:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:10:28.616162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc797449-b1a5-4f8d-8260-7b773532e859 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:10:28.625919 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc797449-b1a5-4f8d-8260-7b773532e859 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:10:28.631544 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa80f87-5e2f-4a63-8175-b42c2729cdf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:10:28.632172 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa80f87-5e2f-4a63-8175-b42c2729cdf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:10:28.632459 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa80f87-5e2f-4a63-8175-b42c2729cdf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:10:28.632739 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa80f87-5e2f-4a63-8175-b42c2729cdf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:10:28.633403 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa80f87-5e2f-4a63-8175-b42c2729cdf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:10:28.635327 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa80f87-5e2f-4a63-8175-b42c2729cdf1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:10:28.730011 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1444/2900] 10.222.0.97 () {60 vars in 1153 bytes} [Tue Jul 26 15:10:28 2022] POST /identity/v3/projects => generated 381 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:10:28.734183 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c64fb-6c5e-477d-b4da-67095c096c95 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:10:28.744753 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c64fb-6c5e-477d-b4da-67095c096c95 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:10:28.804129 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a587cc-5ba3-41df-86d4-b7456731a432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:10:28.804744 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a587cc-5ba3-41df-86d4-b7456731a432 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:10:28.805028 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a587cc-5ba3-41df-86d4-b7456731a432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:10:28.805305 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a587cc-5ba3-41df-86d4-b7456731a432 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:10:28.805962 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a587cc-5ba3-41df-86d4-b7456731a432 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:10:28.807918 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a587cc-5ba3-41df-86d4-b7456731a432 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:10:28.917139 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:10:28.923943 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1457/2901] 10.222.0.97 () {60 vars in 1144 bytes} [Tue Jul 26 15:10:28 2022] POST /identity/v3/users => generated 291 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:10:28.929138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa80f87-5e2f-4a63-8175-b42c2729cdf1 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:10:28.938724 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa80f87-5e2f-4a63-8175-b42c2729cdf1 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:10:28.943702 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd630b86-1c2f-487b-a7bf-f94fe1e9bb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:10:28.944344 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd630b86-1c2f-487b-a7bf-f94fe1e9bb4f admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:10:28.944619 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd630b86-1c2f-487b-a7bf-f94fe1e9bb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:10:28.944891 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd630b86-1c2f-487b-a7bf-f94fe1e9bb4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:10:28.945571 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd630b86-1c2f-487b-a7bf-f94fe1e9bb4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:10:28.947146 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd630b86-1c2f-487b-a7bf-f94fe1e9bb4f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:10:28.957006 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1445/2902] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:10:28.966296 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a587cc-5ba3-41df-86d4-b7456731a432 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:10:28.977256 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a587cc-5ba3-41df-86d4-b7456731a432 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:10:28.982600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290b12c0-6455-4d53-964a-04f6768e26e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:10:28.983635 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b12c0-6455-4d53-964a-04f6768e26e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:10:28.983924 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b12c0-6455-4d53-964a-04f6768e26e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:10:28.984199 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b12c0-6455-4d53-964a-04f6768e26e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:10:28.984892 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290b12c0-6455-4d53-964a-04f6768e26e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:10:28.986564 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290b12c0-6455-4d53-964a-04f6768e26e5 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:10:29.000068 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1458/2903] 10.222.0.97 () {58 vars in 1122 bytes} [Tue Jul 26 15:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:10:29.007866 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd630b86-1c2f-487b-a7bf-f94fe1e9bb4f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:10:29.017848 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd630b86-1c2f-487b-a7bf-f94fe1e9bb4f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:10:29.022707 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd329a-aeb5-4c61-ae98-988718733ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:10:29.023359 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd329a-aeb5-4c61-ae98-988718733ab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:10:29.023674 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd329a-aeb5-4c61-ae98-988718733ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:10:29.023949 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd329a-aeb5-4c61-ae98-988718733ab5 admin admin] PATH_INFO: `/v3/projects/8eeedf7fb2d847f685b76ec5c52755c4/users/8169f0b72007448b805e4447cc835e0f/roles/e033961a9363456a8510c34414dd839e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:10:29.039372 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbd329a-aeb5-4c61-ae98-988718733ab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eeedf7fb2d847f685b76ec5c52755c4, user_id=8169f0b72007448b805e4447cc835e0f, role_id=e033961a9363456a8510c34414dd839e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:10:29.042133 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbd329a-aeb5-4c61-ae98-988718733ab5 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:10:29.045487 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:10:29.218367 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1446/2904] 10.222.0.97 () {60 vars in 1483 bytes} [Tue Jul 26 15:10:29 2022] PUT /identity/v3/projects/8eeedf7fb2d847f685b76ec5c52755c4/users/8169f0b72007448b805e4447cc835e0f/roles/e033961a9363456a8510c34414dd839e => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:10:29.223929 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e4eae-0ca2-4631-ac64-62b2e605f072 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:10:29.224217 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e4eae-0ca2-4631-ac64-62b2e605f072 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:10:29.224496 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e4eae-0ca2-4631-ac64-62b2e605f072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:10:29.244486 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-080e4eae-0ca2-4631-ac64-62b2e605f072 None None] MFA Rules not processed for user `11071bdab076477385007310e6ab9852`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:10:29.245623 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-080e4eae-0ca2-4631-ac64-62b2e605f072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:10:29.305335 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1459/2905] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:10:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:10:29.315200 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd329a-aeb5-4c61-ae98-988718733ab5 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:10:29.317030 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-adbd329a-aeb5-4c61-ae98-988718733ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:10:29.372704 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd329a-aeb5-4c61-ae98-988718733ab5 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:10:29.376344 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbdefa2-58dd-4bad-8f7f-365d154fe28e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:10:29.377044 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdefa2-58dd-4bad-8f7f-365d154fe28e service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:10:29.377325 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdefa2-58dd-4bad-8f7f-365d154fe28e service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:10:29.377602 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdefa2-58dd-4bad-8f7f-365d154fe28e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:10:29.386804 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbdefa2-58dd-4bad-8f7f-365d154fe28e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:10:29.388331 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbdefa2-58dd-4bad-8f7f-365d154fe28e service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:10:29.398689 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1447/2906] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:10:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:10:30.838309 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434bd9a-4156-4b18-bc55-e28c3a1277e3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:10:30.844077 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434bd9a-4156-4b18-bc55-e28c3a1277e3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:10:30.844077 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434bd9a-4156-4b18-bc55-e28c3a1277e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:10:30.867624 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a434bd9a-4156-4b18-bc55-e28c3a1277e3 None None] MFA Rules not processed for user `8169f0b72007448b805e4447cc835e0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:10:30.868857 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a434bd9a-4156-4b18-bc55-e28c3a1277e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:10:30.938249 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1460/2907] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:10:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:10:30.949504 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbdefa2-58dd-4bad-8f7f-365d154fe28e service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:10:30.950896 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ccbdefa2-58dd-4bad-8f7f-365d154fe28e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:10:31.010117 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbdefa2-58dd-4bad-8f7f-365d154fe28e service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:10:31.013657 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4daf28f-15ed-4dac-9ffa-7ecb93faedf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad45b307edc415fa6185a82af64ede4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:10:31.014269 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4daf28f-15ed-4dac-9ffa-7ecb93faedf2 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:10:31.014554 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4daf28f-15ed-4dac-9ffa-7ecb93faedf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:10:31.014832 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4daf28f-15ed-4dac-9ffa-7ecb93faedf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:10:31.023750 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4daf28f-15ed-4dac-9ffa-7ecb93faedf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:10:31.025208 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4daf28f-15ed-4dac-9ffa-7ecb93faedf2 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:10:31.034161 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1448/2908] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:10:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:10:33.045239 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40c3ae-1da8-4568-b9bf-df7351123d26 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:10:33.045811 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40c3ae-1da8-4568-b9bf-df7351123d26 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:10:33.046409 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40c3ae-1da8-4568-b9bf-df7351123d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:10:33.065179 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-be40c3ae-1da8-4568-b9bf-df7351123d26 None None] MFA Rules not processed for user `8169f0b72007448b805e4447cc835e0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:10:33.066391 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-be40c3ae-1da8-4568-b9bf-df7351123d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:10:33.073318 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1461/2909] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:10:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:10:33.313087 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4daf28f-15ed-4dac-9ffa-7ecb93faedf2 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:10:33.323966 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4daf28f-15ed-4dac-9ffa-7ecb93faedf2 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:10:33.327645 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77a710-e17a-479e-861a-e949d3446529 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:10:33.328237 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77a710-e17a-479e-861a-e949d3446529 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:10:33.328511 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77a710-e17a-479e-861a-e949d3446529 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:10:33.328801 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77a710-e17a-479e-861a-e949d3446529 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:10:33.330624 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3c77a710-e17a-479e-861a-e949d3446529 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:10:33.340724 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c77a710-e17a-479e-861a-e949d3446529 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:10:33.342132 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c77a710-e17a-479e-861a-e949d3446529 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:10:33.350317 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1449/2910] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:10:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:10:33.775067 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40c3ae-1da8-4568-b9bf-df7351123d26 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:10:33.776585 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-be40c3ae-1da8-4568-b9bf-df7351123d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:10:33.854022 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:10:33.868982 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40c3ae-1da8-4568-b9bf-df7351123d26 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:10:33.873947 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5daa18-da53-4638-aebd-1e5c526a201c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '252c2d2939134416a3e8efb254f03c5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:10:33.874563 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5daa18-da53-4638-aebd-1e5c526a201c service placement] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:10:33.874843 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5daa18-da53-4638-aebd-1e5c526a201c service placement] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:10:33.875121 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5daa18-da53-4638-aebd-1e5c526a201c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:10:33.876878 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0c5daa18-da53-4638-aebd-1e5c526a201c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:10:33.888088 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5daa18-da53-4638-aebd-1e5c526a201c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:10:33.889739 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5daa18-da53-4638-aebd-1e5c526a201c service placement] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:10:33.900341 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1462/2911] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:10:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:10:43.346587 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:INSERT=1,keystone:SELECT=60 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:10:43.905124 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=9 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:10:44.583987 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77a710-e17a-479e-861a-e949d3446529 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:10:44.585647 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3c77a710-e17a-479e-861a-e949d3446529 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:10:44.686912 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77a710-e17a-479e-861a-e949d3446529 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:10:44.694319 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c1707-7a5b-42d2-a7d3-71b30c102925 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266abef6cda5495b897170f92043aa03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:10:44.694319 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c1707-7a5b-42d2-a7d3-71b30c102925 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:10:44.694319 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c1707-7a5b-42d2-a7d3-71b30c102925 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:10:44.694319 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c1707-7a5b-42d2-a7d3-71b30c102925 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:10:44.696209 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d69c1707-7a5b-42d2-a7d3-71b30c102925 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:10:44.709491 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c1707-7a5b-42d2-a7d3-71b30c102925 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:10:44.711099 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c1707-7a5b-42d2-a7d3-71b30c102925 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:10:44.722014 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1450/2912] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:10:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:10:54.717717 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=46 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:10:59.982545 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5daa18-da53-4638-aebd-1e5c526a201c service placement] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:10:59.993818 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5daa18-da53-4638-aebd-1e5c526a201c service placement] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:10:59.998903 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145260e8-72ca-402e-9442-64b5d52d7417 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '252c2d2939134416a3e8efb254f03c5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:10:59.999710 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145260e8-72ca-402e-9442-64b5d52d7417 service placement] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:11:00.000006 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145260e8-72ca-402e-9442-64b5d52d7417 service placement] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:11:00.000267 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145260e8-72ca-402e-9442-64b5d52d7417 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:11:00.002032 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-145260e8-72ca-402e-9442-64b5d52d7417 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:11:00.014202 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145260e8-72ca-402e-9442-64b5d52d7417 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:11:00.015821 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145260e8-72ca-402e-9442-64b5d52d7417 service placement] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:11:00.026509 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1463/2913] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:10:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:11:10.021942 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=6 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:12:00.346687 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c5fe5-6f7e-4246-aac7-c8302e517f30 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:12:00.347255 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c5fe5-6f7e-4246-aac7-c8302e517f30 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:12:00.347255 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c5fe5-6f7e-4246-aac7-c8302e517f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:12:00.367608 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-bb2c5fe5-6f7e-4246-aac7-c8302e517f30 None None] MFA Rules not processed for user `8169f0b72007448b805e4447cc835e0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:12:00.368833 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-bb2c5fe5-6f7e-4246-aac7-c8302e517f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:12:00.375891 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1451/2914] 10.222.0.97 () {58 vars in 958 bytes} [Tue Jul 26 15:12:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:12:00.622681 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145260e8-72ca-402e-9442-64b5d52d7417 service placement] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:12:00.627336 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-145260e8-72ca-402e-9442-64b5d52d7417 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:12:00.642790 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145260e8-72ca-402e-9442-64b5d52d7417 service placement] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:12:00.647130 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac8d0a-4493-4df5-a1f5-f7b76b331657 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:12:00.647792 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac8d0a-4493-4df5-a1f5-f7b76b331657 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:12:00.648074 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac8d0a-4493-4df5-a1f5-f7b76b331657 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:12:00.648343 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac8d0a-4493-4df5-a1f5-f7b76b331657 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:12:00.650396 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-63ac8d0a-4493-4df5-a1f5-f7b76b331657 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:12:00.662641 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac8d0a-4493-4df5-a1f5-f7b76b331657 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:12:00.664780 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac8d0a-4493-4df5-a1f5-f7b76b331657 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:12:00.674575 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1464/2915] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:12:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:12:01.226212 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2c5fe5-6f7e-4246-aac7-c8302e517f30 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:12:01.236993 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2c5fe5-6f7e-4246-aac7-c8302e517f30 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:12:01.245138 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a234d4-5427-4913-a8bf-09110b2fa782 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:12:01.245811 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a234d4-5427-4913-a8bf-09110b2fa782 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:12:01.246162 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a234d4-5427-4913-a8bf-09110b2fa782 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:12:01.246460 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a234d4-5427-4913-a8bf-09110b2fa782 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:12:01.248484 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c8a234d4-5427-4913-a8bf-09110b2fa782 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:12:01.267773 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a234d4-5427-4913-a8bf-09110b2fa782 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:12:01.270477 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a234d4-5427-4913-a8bf-09110b2fa782 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:12:01.283559 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1452/2916] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:12:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:12:02.418071 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac8d0a-4493-4df5-a1f5-f7b76b331657 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:12:02.426920 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac8d0a-4493-4df5-a1f5-f7b76b331657 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:12:02.432228 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918407d-6104-45bb-a0a2-d3116bf40392 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81180fb4ff04c1186a21e43fb597a91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:12:02.432845 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918407d-6104-45bb-a0a2-d3116bf40392 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:12:02.433132 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918407d-6104-45bb-a0a2-d3116bf40392 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:12:02.433415 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918407d-6104-45bb-a0a2-d3116bf40392 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:12:02.435276 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d918407d-6104-45bb-a0a2-d3116bf40392 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:12:02.449794 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d918407d-6104-45bb-a0a2-d3116bf40392 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:12:02.451237 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d918407d-6104-45bb-a0a2-d3116bf40392 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:12:02.463359 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1465/2917] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:12:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:12:11.280094 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=9 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:12:12.458241 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=12 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:18.342425 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a234d4-5427-4913-a8bf-09110b2fa782 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:18.349086 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c8a234d4-5427-4913-a8bf-09110b2fa782 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:18.358009 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a234d4-5427-4913-a8bf-09110b2fa782 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:18.363243 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7cbc8a-16b7-44d9-bed0-af36f50507d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '252c2d2939134416a3e8efb254f03c5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:18.364021 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7cbc8a-16b7-44d9-bed0-af36f50507d3 service placement] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:18.364309 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7cbc8a-16b7-44d9-bed0-af36f50507d3 service placement] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:18.364722 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7cbc8a-16b7-44d9-bed0-af36f50507d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:18.366508 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ab7cbc8a-16b7-44d9-bed0-af36f50507d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:18.379426 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7cbc8a-16b7-44d9-bed0-af36f50507d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:18.381060 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7cbc8a-16b7-44d9-bed0-af36f50507d3 service placement] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:18.391420 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1453/2918] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:13:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:13:18.494851 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918407d-6104-45bb-a0a2-d3116bf40392 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:18.508568 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918407d-6104-45bb-a0a2-d3116bf40392 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:18.510709 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e8d14-633d-4228-bd7c-6a10c78b8652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:18.511426 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e8d14-633d-4228-bd7c-6a10c78b8652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:18.511833 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e8d14-633d-4228-bd7c-6a10c78b8652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:18.512125 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e8d14-633d-4228-bd7c-6a10c78b8652 admin admin] PATH_INFO: `/v3/users/8169f0b72007448b805e4447cc835e0f` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:18.513257 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766e8d14-633d-4228-bd7c-6a10c78b8652 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8169f0b72007448b805e4447cc835e0f)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:18.515262 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766e8d14-633d-4228-bd7c-6a10c78b8652 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:18.753270 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-766e8d14-633d-4228-bd7c-6a10c78b8652 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8169f0b72007448b805e4447cc835e0f'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:18.753896 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-766e8d14-633d-4228-bd7c-6a10c78b8652 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8169f0b72007448b805e4447cc835e0f'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:18.766052 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-766e8d14-633d-4228-bd7c-6a10c78b8652 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8169f0b72007448b805e4447cc835e0f'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:18.772933 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-766e8d14-633d-4228-bd7c-6a10c78b8652 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8169f0b72007448b805e4447cc835e0f'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:18.773625 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-766e8d14-633d-4228-bd7c-6a10c78b8652 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8169f0b72007448b805e4447cc835e0f'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:18.865678 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1466/2919] 10.222.0.97 () {58 vars in 1224 bytes} [Tue Jul 26 15:13:18 2022] DELETE /identity/v3/users/8169f0b72007448b805e4447cc835e0f => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:19.138792 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7cbc8a-16b7-44d9-bed0-af36f50507d3 service placement] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:19.157498 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7cbc8a-16b7-44d9-bed0-af36f50507d3 service placement] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:19.240222 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14824691-220e-416e-99fd-1cbb5554706c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11071bdab076477385007310e6ab9852', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '011581630bbe4c60a042395d12032595', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:19.241198 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14824691-220e-416e-99fd-1cbb5554706c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:19.241654 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14824691-220e-416e-99fd-1cbb5554706c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:19.242103 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14824691-220e-416e-99fd-1cbb5554706c admin admin] PATH_INFO: `/v3/projects/8eeedf7fb2d847f685b76ec5c52755c4` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:19.243954 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14824691-220e-416e-99fd-1cbb5554706c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eeedf7fb2d847f685b76ec5c52755c4)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:19.247215 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14824691-220e-416e-99fd-1cbb5554706c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:19.386211 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-14824691-220e-416e-99fd-1cbb5554706c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eeedf7fb2d847f685b76ec5c52755c4', 'request_id': 'req-14824691-220e-416e-99fd-1cbb5554706c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:19.391717 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-14824691-220e-416e-99fd-1cbb5554706c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eeedf7fb2d847f685b76ec5c52755c4', 'request_id': 'req-14824691-220e-416e-99fd-1cbb5554706c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:19.471233 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-14824691-220e-416e-99fd-1cbb5554706c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eeedf7fb2d847f685b76ec5c52755c4', 'request_id': 'req-14824691-220e-416e-99fd-1cbb5554706c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:19.486345 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-14824691-220e-416e-99fd-1cbb5554706c admin admin] The token cache is being invalidate because project 8eeedf7fb2d847f685b76ec5c52755c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:13:19.487136 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-14824691-220e-416e-99fd-1cbb5554706c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:19.487860 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-14824691-220e-416e-99fd-1cbb5554706c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:19.489438 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1454/2920] 10.222.0.97 () {58 vars in 1233 bytes} [Tue Jul 26 15:13:19 2022] DELETE /identity/v3/projects/8eeedf7fb2d847f685b76ec5c52755c4 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:28.861493 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:29.480973 dv-cinder-851030-1-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:03.937891 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e8d14-633d-4228-bd7c-6a10c78b8652 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:03.939349 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-766e8d14-633d-4228-bd7c-6a10c78b8652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:04.013600 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e8d14-633d-4228-bd7c-6a10c78b8652 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:04.074155 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8437756b-0937-43a2-b555-6c552bbfd541 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '252c2d2939134416a3e8efb254f03c5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4c8da3011294783b2180ade1bfd9366', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:04.074783 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437756b-0937-43a2-b555-6c552bbfd541 service placement] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:04.075072 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437756b-0937-43a2-b555-6c552bbfd541 service placement] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:04.075423 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437756b-0937-43a2-b555-6c552bbfd541 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:04.077149 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8437756b-0937-43a2-b555-6c552bbfd541 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:04.089414 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8437756b-0937-43a2-b555-6c552bbfd541 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:04.091062 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8437756b-0937-43a2-b555-6c552bbfd541 service placement] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:04.101699 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1467/2921] 10.222.0.97 () {60 vars in 1365 bytes} [Tue Jul 26 15:16:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:16:14.097360 dv-cinder-851030-1-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=58 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}